4. Outreach Programs

Federal health agencies—HRSA, CDC, and SAMHSA—have placed greater emphasis on identifying HIV infected persons and linking them to care. Outlined below, programs can be categorized by where they focus on in care/out-of-care continuum, such as the newly-tested or re-engagement of those who have fallen out-of-care.  They can also be viewed by the interventions they use, such as street outreach, engagement in clinical and corrections settings, and use of case managers and peers to conduct intensive interventions and support for clients.

Types of Outreach Programs  TOP

• Short-Term Case Management to Link Newly Tested to Care.  TOP
  CDC, the major public funder of HIV prevention work and HIV counseling and testing services in particular, refocused much of its prevention work in 2004 to more aggressively do HIV testing and work with diagnosed persons through case management to link them to care.  This in part was a response to a 1999 Journal on AIDS (JAIDS) study that 40 percent of those who know their status reportedly delayed entry into care for over one year. CDC’s Antiretroviral Treatment and Access Studies, called ARTAS, uses short-term case management to engage newly-diagnosed individuals into at least two primary care visits within the first year of diagnosis. 
  
  Characteristics of the intervention include work to overcome barriers to care (e.g., fear, lack of readiness, lack of knowledge about services) and active work to link the client to services (e.g., accompany clients to appointments, transportation to the first medical visit).  As many as five contacts with an ARTAS case manager are made over a 90 day period.  Findings to date showed that 78 percent of those having ARTAS case management contact stayed in care six months longer as compared to just 60 percent who got a passive referral to care.  When extending the period out to 12 months, 65 percent of ARTAS subjects were still in care while only 49 percent of passive referrals were so linked.  See CDC Website.  Additionally, cross-sectional and longitudinal evaluation of barriers indicates at least some of the effect of case management in helping people get in care is due to a reduction in barriers to care.
   
• Social Networking: Testing the At-Risk  TOP
  CDC-funded social networking, which enlists HIV-positive and high-risk HIV negative persons in communities of color to identify and recruit high risk individuals from their social, sexual, or drug-using networks.  They then help link them to HIV counseling and testing services (in the field or in test sites) and refer them to care if infected.  If contacts are at risk of infection, referrals are made for medical care, prevention, and other services as appropriate, such as substance abuse treatment. Social networkers can work with peers alone or with provider assistance. 
  
  Preliminary findings are in Use of Social Networks to Identify Persons with Undiagnosed HIV Infection, Seven U.S. Cities, October 2003-September 2004. MMWR. June 24, 2005 , 54(24);601-605. 
  
  Among the barriers identified is lack of understanding of testing in social networks as well as staff turnover and the difficulty of training staff in such areas as interviewing skills with peers and garnering information about sexual partners.  Coordination difficulties included referral tracking systems and project monitoring.  In hindsight, the program would have developed a training curriculum, trained staff more, and recognized that enlisting newly infected recruiters is difficult.  Among additional insights: case management makes the process work more smoothly as do non-cash incentives, recruiter referrals, and a referral tracking system to ensure that people with HIV get linked to medical services.
  
• Health System Navigation: Helping Clients Access Care and Building Agency Capacity to Support Clients.   TOP
  Under this two-fold design, extensive supportive services are first provided to clients to help them navigate the health care system.  Secondly, infrastructures among multiple service agencies are built to—in turn—support clients who are referred to them.  Fenway Community Health operates such a program in partnership with six agencies in Boston.  Services to clients include outreach to persons who are HIV+, unstable or out-of-care and referral to HIV testing through CDC’s social networks model.  The theoretical underpinnings of the program are the Popular Opinion Leader model, Diffusion of Innovations, and Transtheoretical Stages of Change.
  
  In Phase 1, Fenway partnered with three community-based organizations to engage and retain their clients living with HIV in HIV medical care. Although many participants were connected to HIV medical care, their care was unstable over time. In Phase 2, the program focuses more explicitly on identifying individuals who are not stable in care.  A screening tool is used to help identify unstable and out-of-care clients.  Subsequent engagement is undertaken to help clients engage in care.  Health Systems Navigators (HSNs) conduct brief assessments, develop client-driven action plans, and work with clients to achieve their goals. HSNs are not based within a single organization but rather conduct outreach and provide services in the community itself, meeting their clients at home or service agencies, and accompanying them to appointments.
  
  Home-visit services frequently entail working not only with individual clients but also with family members and support systems. Appointment adherence is supported through telephone reminders, chaperone and transportation to appointments, and follow-up calls when appointments are missed. The program also chaperones clients to non-medical appointments, such as case management, custody or court hearings, and financial assistance determinations. Directly-observed therapy is provided to clients who need this additional level of medication adherence support.
  
  The skill sets of an HSN in providing the above assistance can either stand alone as an HSN job description or be incorporated into the job descriptions of a peer advocate, transitional case manager, or outreach worker. The HSN intervention is designed to be time limited and focused on helping people become stable in care through the establishment of more permanent relationships with culturally competent medical providers and case managers. Thus, the HSN, working to complement case management services, may overlap with case management to a limited degree—such as making referrals—but only until such time as the client is receiving consistent case management.
  
  One challenge for the program is training HSN staff. They work for different organizations, making it difficult to impart common understanding of project goals, the program model, practical aspects of their jobs, and reasons for project evaluation.  In Phase 2, the Fenway Institute convened a Training Academy for staff and their supervisors from programs throughout Boston with responsibilities similar to those of HSNs to learn about the intervention. Collaborative efforts are underway to create a sustainable HSN program, where staff of multiple agencies citywide work together to address the varied needs of people who are not receiving consistent HIV-related medical care.
  
• Traditional and Media-Based Outreach to Increase Testing.   TOP
  This small project of AIDGwinnett, a CARE Act grantee, is located in a rural area of Georgia and operates with a mix of media and traditional outreach to expand awareness of HIV testing and counseling in the community.  Specific work includes expansion of HIV testing and counseling in nontraditional venues, training additional staff (e.g., volunteer outreach workers to target higher risk venues), and targeting of testing in high-risk areas using mapping technology.  Migrant workers are among the populations being targeted. Also, incentives (e.g., providing rapid testing supplies to clients and give-aways) are used to enhance receptivity to testing.  Public information (e.g., spot announcements in minority radio) and Web site publicity is used to expand awareness of testing services. These strategies appear to have paid off, with an increase of 900 persons coming in for testing in the first year.
  
  Evaluation efforts to determine the efficacy of testing outreach include use of CaseTrakker software to determine which clients enter care.  This is used as part of a client management program.  All client encounters are documented and other process data are collected (e.g., number of contacts, number of tests, demographics, incentives distributed).  The program also tracks, at the time of testing, to determine how individuals learned about testing. 
  
• Use of Multiple Techniques to Engage/Retain PLWH in Care.   TOP
  Cascade AIDS Project in Portland, Oregon, uses a mix of techniques (e.g., social networking, peer referral, case management) to do both outreach and retain people in care. This program, called CARELink, uses an array of interventions, from individual work in nontraditional venues to small group sessions. 
  
  Work with clients is based on the transtheoretical model and stages of change and seeks to engage them in readiness to enroll/engage in care and develop individualized plans to move them along the readiness continuum.  As such, their work seeks to build relationships with clients and increase their skills-building around HIV, accessing services, and self-advocacy. 
  
  The project focuses on those thought to be at high risk for dropping out or follow-up with those who have dropped out-of-care.  Social networks are used to bring out-of-care PLWH into care through peer referrals. The project also works with external case management providers to identify the out-of-care/at-risk and bring them back into care.  To assess those at highest risk of dropping out, continual assessment is done of client barriers and the status of ongoing case management with clients. 
  
  The two target populations include the hard-to-reach (defined as persons who are homeless, addicted to drugs or have mental illness) and Latinos.  To date, project findings are that PLWH with multiple barriers need more intensive harm reduction based services to remain in HIV care.  The availability of other support services, especially housing, is seen as essential to getting people into care.  Notably, the most success in getting clients into care is seen with those reached earlier in their diagnosis.  Finally, CARELink sees the need for additional research in using motivational interviewing and other cognitive behavioral interventions with hard-to-reach populations to improve future interventions with out-of-care populations.
  
• Street Outreach: Community Health Care Van.   TOP
  The Yale AIDS Program is a street-based program built around a mobile van that delivers an array of medical and supportive services to injection drug users (IDUs) in inner-city New Haven.  It is staffed with a clinician, case manager, HIV counselor/tester, and outreach workers who mirror the target population groups.  Staff are immediately accessible via cell phone, and come to the van as soon as required to meet a client’s expressed needs. While free clinical care is available to all those interested, the van targets out-of-care injection drug users (IDUs) living with HIV, particularly people of color.  Services include purified protein derivative (PPD), sexually transmitted disease (STD), and HIV testing, physicals for drug treatment, hepatitis B vaccine (HBV), primary care with free medicine, and on-site drug treatment with buprenorphine.  This menu of services means the van is recognized as a primary care vehicle, not “the HIV van,” which increases client comfort and utilization.
  
  Among the lessons learned to date is the importance of having a non-judgmental attitude toward clients and focusing on readiness to care for clients who find it challenging to focus on medical care given such competing concerns as housing and substance abuse.  Building trust is seen as crucial to the work, as is avoiding barriers to care through, for example, flexible hours and bilingual staff who are readily accessible to clients.
   
• Transitional Case Management to Retain Ex-Offenders in Care.   TOP
  Project Bridge, a program of Miriam Hospital, targets prisoners in Providence, Rhode Island to facilitate re-entry to the community and help them remain in care, post-release.  The intervention starts 90 days prior to release and extends out, generally to 18 months after release.  Work focuses on assessing client need; establishing a treatment plan; facilitating medical care, medications, and insurance coverage; and increasing social stability. This happens through the provision of intensive case management prior to and following prison release.  Community based two-person teams of social workers are assigned to each case and start out with daily contact in the first month after release.  This tapers off over time, hopefully as a consequence of increased client adoption of positive care behaviors. Contacts are weekly at weeks 5-12 and monthly thereafter.  Staff accompany clients to medical exams and provide a host of other supports—from referrals to counseling to adherence support.
  
  A host of evaluation findings to date show high levels of receipt of ongoing care. Ninety-five percent of eligible people enrolled in the program and 90 percent had a clinic visit their first month and 83 percent have at least one clinic visit in six months. Ninety-four percent had a direct contact in the past 30 days.  Most clients see the program through: 84 percent completed 18-month enrollment and 87 percent had clinic visits at 24 months.  Clients had missed, on average, 1-2 clinic appointments per six months.  There was a slight improvement in TOFHLA (Test of Functional Health Literacy in Adults)scores at 12 months.  Finally, medical outcomes also improved in terms of increased CD4 count and decreased viral load.
  
  Additional evaluation trends show high levels of receipt of services intended to enhance receipt of HIV care: 77 percent of referrals result in services received.  Homeless rates declined (there were 17 percent on the streets at baseline and only 4 percent at completion).  Additionally, 58 percent were uninsured at baseline and 100 percent at 12-month follow-up.  All clients received transportation, food, and benefits assistance at six months.  Finally, 57 percent received mental health care and 81 percent got substance abuse treatment.
  
• Predicting Drop-Outs and Preventing Them.   TOP
  The methodology used by this SPNS program at Washington DC’s Whitman-Walker Clinic is to screen for clients most likely to drop out of care and to then link them to a retention care coordinator, or RCC. The screening criteria were developed by assessing predictors of non-retention in care as seen in their clinic.  Predictive factors for having a greater than 75 percent chance of non-retention in primary medical care were identified as: race/ethnicity, years clean, substance abuse, and employment status. 
  
  Those meeting these criteria were referred to one of two arms in the Phase II study to determine what works in retaining people in care.  The arms differ in the level of intensity of the RCCs work over 12 months.  At a minimum, both arms receive courtesy call reminders, transportation assistance, and childcare assistance.  The higher intensity arm additionally includes no-show follow-up, chaperone to appointments, and system navigation education.
  
  Data were collected from chart reviews and intensive interviews with clients to determine what processes and factors influenced retention. The no-show rate for clients, before the intervention, was 24.5 percent; after at least six months in the study, the no-show rate dropped to 16.5 percent.  Additionally, there were high levels of provider and client acceptance of the intensity of the interventions. After working with the RCCs, providers recognized the benefits of their involvement and subsequently made referrals to the program.  For clients, RCC support is reportedly accepted and not seen as intrusive.
  
• Health Promoters for Adherence Support.   TOP
  The Prevention and Access to Care and Treatment (PACT) Project, based in Boston, uses health promoters to improve access to care for HIV patients in Boston’s inner city who are marginalized on several fronts: low-income, minority, and reliant on supplemental security income (SSI), Medicaid, and other public payers. The program receives referrals from providers with non-adherent patients or clients with substance use. Also referred are individuals whose CD4 count is below 350, or whose viral load is over 1000 for the past 12 months.
  
  Intervention is provided through an empowerment model, with the goal of helping non-adherent or at-risk clients move to self-management of their medication schedules. Upon intake, client needs and barriers to adherence are assessed, and the level of engagement is determined. Low intensity includes monitored self-administration with monthly health promotion.  Moderate intensity entails weekly health promotion, while high intensity involves directly observed therapy plus additional services.  Clients are evaluated at three and nine months and the level of intervention is adjusted if needed. For example, if adherence is not improved at three months, they offer direct observation of therapy to the client until such support is not needed.
  
  Health promoters conduct intensive engagement with clients, in conjunction with physicians, social scientists, social workers, and case managers.  Health promoters are recruited from the affected community—some are even past PACT participants—and receive extensive curricula and field-based training at the onset and ongoing monitoring and supervision.
  
  Promoters conduct HIV education, adherence support, counseling, and other health education.  They also translate treatment recommendations into the home.  They typically accompany clients to appointments, provide home-based support, and facilitate access to and utilization of resources.  They serve as a surrogate support network and sounding board for clients.
  
  Evaluation is ongoing, but PACT reports on what medical cost savings it believes are realized from the program.

Common Features of Engagement Programs  TOP

Programs vary in their target populations and techniques used to engage and retain people in care.  In part, strategies vary based on where clients fall along the continuum.  To illustrate, programs targeting the never tested and never in care include broad-based street outreach and use of recruiters or peers, such as identifying new testers through social networking.  Programs targeting sporadic users of care within their own clinic programs use techniques like intensive case management and clinic-based support.  This involves, for example, coordinating, as a routine part of clinical care, medical and case management providers to learn who is out of care or is missing appointments. Extensive tracking involves work to find clients by contacting family, friends, place of employment, other agencies and other venues.

Other programs target sporadic users of care by identifying clients at highest risk of falling out of care as part of intake or assessments. Clients who meet certain characteristics or profiles are identified and offered additional client education and support services.

Regardless of approach, seasoned outreach/retention programs tend to have common features in the realms of client-level, provider-level, and systems-level.

Client Level Features  TOP

Client features reflect the need for engagement to build relationships with clients in helping them get in care and stay there, especially given that high proportions of PLWH have complex and multiple needs such as substance abuse and mental health issues. Lack of insurance, homelessness, unemployment, incarceration, and domestic violence are among the other barriers. Among the features to address these challenges:

• Intensive Services and Support. Those who have a hard time staying in care need extensive help and support, which can be staff-intensive and long-term. For example, a corrections-focused project in Rhode Island initiates transitional case management services for soon-to be released prisoners 90 days prior to discharge and follows them for 18 months post-discharge. Client contact increases from daily during the first month to weekly over the next four months, and finally monthly over the next year. The mean number of contacts is 20 direct contacts and 35 phone contacts in the first six months.  Similarly, SPNS project client contacts range from 2-22 encounters over six months and average 10 encounters. Many SPNS clients require up to 18 months before they engage in care.  
  
  The intensity of services is driven not only by frequency but also the type of services provided. Examples include: home-based visits; working with families/support systems; and appointment adherence supported through telephone reminders, chaperone services, transportation to appointments, and follow-up calls when appointments are missed. Several programs chaperone clients to non-medical appointments, such as case management, custody or court hearings, and financial assistance determinations. Directly-observed therapy is also provided to support adherence.
  
  The goal of all this support is to help clients achieve greater levels of self-management. Clients may move from a higher to a lower level of service intervention as they demonstrate their capacity to negotiate care, keep appointments, and adhere to medication.
  
• Assessment of Client Needs. Outreach programs often assess and screen clients at the outset to determine what they need to help them stay in care. Client assessments—of needs and care readiness—differ somewhat but generally include: determination of client needs for mental health, substance abuse, housing, entitlement, legal, medication and medical care, transportation, child care and other support services; barriers to treatment adherence; and assessment of home environment and supports.
  
  Many projects also assess health beliefs and health literacy (using the TOFHLA orTest of Functional Health Literacy in Adults).  This is part of a determination of how equipped a client is to manage his or her own care and is part of examining what is called “readiness for care,” with knowledge being just one element to assess.   The behavior change theoretical model has been used to customize client interventions depending on client readiness. Based on their assessment, individual service plans are developed.
  
• Engender Client Trust.   Many clients have had little or no experience with health care systems. This reportedly requires development of trust and building of relationships with individuals as a prerequisite to engaging them in care. First and foremost, this requires being non-judgmental and accepting of clients, regardless of substance use, sexual behaviors, mental health, or other issues or behaviors. Another aspect of trust building is being honest with clients and taking the time to give them complete information. “Being there when needed” tells clients they are important and staff can be counted on. Examples of being readily available include outreach workers and case managers who give their pager numbers to clients and clinicians that are available for calls around the clock.
  
  Making clients comfortable with staff and provider sites is key to prompting client engagement in care. This might entail hiring staff and outreach workers who are demographically and culturally similar to the target subpopulations, such as use of peers as staff. Another strategy is to “look like the population.” At one program, this means no white lab coats; staff dress similar to the clientele and, in effect, learn how to speak their language. Sites also increase client comfort about not being identified as HIV-positive by not placing the terms “HIV” or “AIDS” on their physical facilities. Further clarifying expectations, so clients and staff understand their respective rights and responsibilities, is also helpful in developing trust.
  
• Meet Client Priorities First.  This means recognizing each client’s priority needs and addressing them before tackling HIV care needs. Clients will not always be ready or interested in receiving primary care. Their focus may be on such issues as housing, food, clothing, legal, dental, or substance abuse needs. Providers see such encounters as opportunities to build the trust and bring individuals closer to considering care.
  
• Readiness for Care. Clients are at varying stages of readiness for care. Whereas the substance abuse field has developed models for readiness and behavior change, similar models for entry and retention into HIV care have not been formalized. Some projects have designed strategies to assess client readiness, such as motivational enhancement interviewing to help clients define readiness to enroll/engage in care. From this, an individualized plan is developed to move the client along the readiness continuum. Based on the transtheoretical model and stages of change (Prochaska and DiClemente, 1983), the intervention focuses on the following aspects of the processes of change: client freedom of choice, enhancing client self-confidence by acknowledging their ability to change; establishing rapport and helping build the client-provider relationship; and offering professional advice and brief education.  Another program focuses on building the capacity of community agencies and providers by training them in stages of change and motivational interviewing.
   
• Client Health Beliefs/Health Literacy.  Health beliefs appear to significantly influence clients’ care-seeking behavior, as demonstrated by SPNS and ARTAS projects. Clients who believe they are not sick enough for care or that their faith will help cure their HIV disease are less likely to be in care, according to SPNS data. Assessing health beliefs during outreach, at intake, and at various points during a program intervention helps programs target education and devise interventions to address those beliefs.
  
  Persons who are not knowledgeable about HIV disease, treatment options, and services are, according to outreach project observations, less likely to enroll/engage in health care.  This is an aspect of client health beliefs but differs because it is about changing beliefs and understanding: about building health literacy.  This can range from building client self-advocacy to enhancing understanding of one’s health status so that clients can better manage their health needs.  This can happen through workshops or one-on-one.  One technique used is to conduct patient and family health education (the latter so the family can support the learning) and translation of treatment recommendations into the home.

Provider/Clinic Features  TOP

Provider techniques to engage clients in care include creating a receptive agency environment and enhancing staff skills.  A major finding of engagement programs is that providers, particularly clinicians, may be initially resistant to engagement work but become appreciative of the assistance it provides and often very supportive of its aims.  Following are some of these provider features.

• Get Out Into the Community. Outreach workers by definition get out to various locations to locate clients, but they must go to the places where target populations can be reached and must use the techniques that work best with specific client groups. Outreach venues include bars, barbershops, teen centers, parks, and other public locations. Recruiters and social networkers learn about community gathering spots and go to these venues to find those likely at highest risk for HIV infection.   However, the outreach work that is done in these locations tends to be focused on re-linking people to care, or helping support sporadic users, more so than identifying new clients who are not engaged at all.  Programs report great challenges in identifying the undiagnosed and out-of-care via such outreach work.
  
  Outreach programs are aware of the need to be diligent in locating people who have fallen out-of-care while respecting client personal rights and autonomy. The question was raised: When is client-tracking too intrusive, impinging on client’s freedom? Generally, outreach projects report an obligation to vigorously seek clients who disappear from care. Providers typically try and assess whether a client has fallen out of care for a “reasoned” decision or because of substance abuse.  If the latter, efforts are made to find the client and discuss treatment when the individual’s judgment is clear. Another approach is to inform clients in advance of the actions outreach staff will take to track them down. One project enters into a contract, signed by both the client and project staff, which delineates expectations of the client and exactly how many contacts, to whom, and over what period of time the program will look for them.
  
  Medical providers who deliver care in the community can build linkages to individuals through non-clinic encounters. A clinician who treats adolescents, for example, reports doing in-home teen HIV education/counseling and testing sessions and subsequently getting calls and even clinic visits from youth who were afraid to test in front of their friends. 
  
• Be Welcoming and Accessible. Client comfort with an agency’s clinic or other site may influence their willingness to come, return to, or stay in care. Agency space and staff (especially receptionists) must be welcoming.  Other accessibility features include good phone coverage and call back systems, translation services, and flexible hours. Co-location of services can also improve client access to services, such as co-locating substance abuse with medical care, and counseling and testing in a facility with medical care. ARTAS found that co-location worked better to link clients into care.  However, this is not always essential, and one drawback of such “one stop shopping” is that clients are often obligated to spend large portions of a day going to various services within the same site.
  
• Break Down Physician Resistance.  Caring for complex clients is challenging and can be met with resistance by some clinicians.  For example, the ARTAS project found that 24 percent of physicians surveyed in four cities were resistant to providing care to IDUs, based upon a Likert scale.  In addressing this, programs spend considerable time building clinician buy-in to the outreach program and teaching them its benefits—most notably, the added support it provides to clinicians.  The most powerful way to get clinician buy-in, cited by multiple programs, is when intensive interventions enhance client compliance with care regimens. 
  
• Staffing. Those serving as the lead for engaging people in care go by many titles: social networkers, recruiters, transitional case managers, retention coordinators, and promoters.  Their common mission is to work closely with clients to help them obtain needed services, learn to navigate the system, and access services.
  
• Programs that target those who do not know their HIV status or just learned their status use social networkers and recruiters to find individuals—those identified through referrals or identified as members of high risk networks—to refer them to counseling and testing. Social networkers may also be used to bring those who fell out of care back into care. These positions are typically held by community members.
• Transitional case managers or linkage coordinators generally provide intensive case management, adherence support, and education to clients. They target clients at all stages in the care continuum.  The intervention may be brief (such as with CDC’s ARTAS, which provides five case management visits over 90 days or longer term (such as a program that works with clients for 18 months). The goal is to provide up-front support to help clients enter into care.
• Health promoters and retention coordinators may work with clients at all points in the continuum. They often provide intensive support, client education, systems navigation, and adherence services to engage or re-engage clients in care. They may serve as client advocates and attend medical clinics and other visits with clients.

Programs show flexibility and creativity in use of staff. One program has joint intake teams where their CDC-funded transitional case manager works with their Ryan White CARE Act community-based case manager to deliver client services and facilitate a smooth transition to community-based case management. Another retrained case managers to provide more therapy-based interventions to clients at highest risk of falling out of care.

• Peers and Community Members.  HIV-positive and at-risk community members are utilized in outreach because of their life experiences, which they can use to help relate to clients. HIV-positive and high-risk HIV-negative recruiters are used in social networking projects, which seek to identify and refer associates to counseling and testing services. Among the benefits of using peers: they are effective at translating HIV information and treatment issues into understandable terms, and communicating client concerns to service providers.  However, management of peer staff can be challenging and is not universally embraced by outreach programs.
  
• Staff Training. Training to help staff develop new skills, learn to navigate the health care system, or to work with clients can be extensive. In one program, health promoters receive 56 hours of initial training and additional weekly supervision.  Outreach workers and transitional case managers, especially those traditionally funded by CDC prevention funds, are educated about health care services and community providers, so that they might link clients to care sites or providers of ancillary services. Agencies entering new collaborations with other agencies conduct cross-training on each agency’s role and how staff work together.
  
• Measuring Outcomes.  Programs use varied measures to assess their interventions’ impact on clients.  Most projects evaluate their programs in at least two or more of the following areas:
  
• Number or clients identified and referred to care.  This is of particular interest to models with a component to locate newly diagnosed and out-of-care (social networkers, peers, counseling and testing activities).  
  
• Clients linked to medical care. This may be evidenced by CD4 count (for newly diagnosed) or by information documenting receipt of care in various venues.  For example, CDC’s ARTAS study reported a higher proportion of those receiving case management interventions visited a clinician at least twice over 12 months as compared to individuals who were simply referred to a social worker (64 percent vs. 49 percent).
  
• Clinic visits and appointment keeping.  Ways to examine this include length of time from referral to first visit, frequency of visits according to a predetermined time period (e.g., within the first 30 days or every six months); and missed appointments.  This information can be obtained via self-report or through verification or medical records review.  Record audits are cumbersome and access to records is not always granted. CDC found 93 percent accuracy between self-reports compared to medical records in first six months and 86 percent accuracy after 12 months. Improved referral tracking and follow-up could assist with, for example, forms sent by medical (or case management) providers if clients do not show.
  
• Clinical markers. This can include a determination of presence of a CD4 count/RNA count as an indicator of in care in that providers were testing clients in order to monitor their disease state.  These measures may also be examined in looking for trends, such as review of changes in CD4 and RNA values as a proxy for adherence or improved health. CD4 and RNA measures are more readily available in clinic-based programs than community social service agencies.
  
• Referrals.  Ways to measure this include ancillary service referrals made and provided, especially use of mental health/substance abuse services, given the high level of these needs among clients.
  
• Client health and mental health.  These health indicators may be self-reported or obtained from such measures as an Activities of Daily Life scale or a Depression scale to gauge a client’s mental health.
  
• Health literacy and health beliefs.  Among the tools to measure health literacy is TOFHLA (Test of Functional Health Literacy in Adults). 
  SPNS outreach grantees use a standardized set of evaluative measures. They can be obtained via the SPNS Outreach Web Site.

Systems Features  TOP

Among the systems features used by outreach/care retention programs is collaboration among agencies to facilitate client access to services and achieve more seamless care. Yet another feature is data sharing among community partners to follow clients across systems of care and to demonstrate program outcomes.

• Collaboration. New collaborations include links among outreach and testing referral programs with case management and care programs.  In turn, case management departments are aligned more closely with medical providers (both within and external to their agency and sites). Many programs have strengthened referral relationships with providers of critical support services (e.g., mental health, substance abuse treatment).
  
  Some programs coordinate not only with HIV agencies but across systems, such as a project in a corrections setting for post-discharge clients that works to establish trust and joint post-discharge planning with the corrections agency. By arranging to initiate planning prior to discharge, ex-prisoners have continuity of insurance, treatment and medications upon their release.
  
• Multi-level Collaboration. Collaboration needs to take place at both the agency leadership and staff level. Establishing new referral arrangements and client-information sharing requires
  involvement of agency management.  Agencies must be knowledgeable about one another’s target populations, eligibility and referral procedures, services, and areas of expertise.
  
  Such collaboration can help in many ways: 1) referrals for needed services are expedited; 2) information is shared about client appointment keeping; and 3) client transition from one agency to another (transitional case manager to community case manager) is more seamless. One technique used is multidisciplinary team meetings of physicians, case managers, retention care coordinators and linkage coordinators.  This team approach allows engagement staff to provide unique insights about the client’s needs, barriers, misperceptions or life circumstances that helps tailor client treatment plans and adherence strategies.
  
  To illustrate, a SPNS project established a new partnership of 20 Boston agencies, both to build their capacity to engage clients in care and to improve referrals and systems navigation. The training covers intervention models for engaging persons in care, behavior change models, and service navigation. While the goal is to enable clients to better access services through various service points of access, the training benefits case workers and agency management as well.
  
• Data Sharing. Cross-agency data sharing can be difficult to accomplish due to different tracking methods and incompatible databases. It is most difficult to track clients seen by private providers or other public systems (e.g., Medicaid). Tracking clients from prevention to care is one of the greatest areas of difficulty in social network programs in that they require links across prevention and care agencies. Referral and client tracking across systems is critical to following individual clients and demonstrating program success. 
  
• Community and Provider Education.  Bridging the care/prevention divide has resulted in a melding of such concepts as “outreach,” “care coordination,” and transitional case management.  Many new and unfamiliar positions have subsequently been funded for client engagement and retention (e.g., linkage case managers, retention coordinators), and agencies and staff providing outreach and care services are often not familiar with the terms and roles of staff in doing their work. For example, some case management agencies that work with transitional case managers are unfamiliar with them, including concerns that duplication of work in the field was occurring. 
  
  Agencies may need to educate community service providers on the goals, methods and staffing for newly funded outreach/linkage projects or positions.  Community-based agencies can come to value the linkages that case managers make and become supporters of their efforts to retain clients in care. For example, transitional case managers in one program might help meet clients’ most immediate needs and initiate client education about HIV care and services. This can then facilitate the transfer of clients from programs with limited interventions to those with intensive case management services.
  
  While many agencies lack funding for this type of collaboration, such cross-agency planning, development and education might be addressed through planning bodies under CDC and CARE Act mechanisms, especially in areas with common planning groups or coordination across their bodies.