5.2. Process and Effectiveness Evaluation
5.2.1. Process Evaluation
Process evaluation refers to exploring how a service works. This means understanding the philosophy of Housing First. Ensuring the design of a Housing First service is fully understood, i.e. how the Housing First service is supposed to work is a very important first step in evaluation. A key measure here is the level of fidelity (similarity) to the original Housing First model. Fidelity refers to the core principles of Housing First and the operational detail of the successful original model of Housing First.
5.2.2. Fidelity
Assessing fidelity is the starting point of an evaluation of Housing First. Fidelity refers to how closely a service follows the core principles of Housing First (see Chapter 2). If a service does not follow the core principles, it should not be regarded as Housing First and should not be evaluated as an example of Housing First.
Fidelity measurement can also be described as testing for paradigm or model drift (moving away from the original model), which is a fundamental principle of any service evaluation. This means making sure that the Housing First service being tested is close to the original service design, i.e. that a service has not drifted away from, or was never really close to, the core principles of Housing First. In evaluation, this is very important because it tells the evaluators and anyone hearing about the results of an evaluation whether or not a successful Housing First service, or a Housing First service with problems, had high or low fidelity with the core principles of Housing First. This is important because success or failure may both be heavily influenced by fidelity and it is crucial to understand whether, for example, poor results from a particular Housing First service could be explained by low fidelity. The evidence from Europe so far suggests that success in Housing First is linked to high fidelity with the core principles((Pleace, N. and Bretherton, J. (2013) The Case for Housing First in the European Union: A Critical Evaluation of Concerns about Effectiveness European Journal of Homelessness, 7(2), 21-41 http://housingfirstguide.eu/website/the-case-forhousing-first-in-the-european-union-a-critical-evaluation-of-concerns-about-effectiveness/)). Housing First services that follow the core principles, although they work in European countries with sometimes very different welfare, health, housing and homelessness systems, have all delivered good results in ending homelessness (see Chapter 1).
Fidelity tests exist in North America, are being developed for use in Europe and are also being developed and used in individual European countries. The operational details may vary, e.g. whether or not social housing is used, or whether a service employs an integrated multidisciplinary team, intensive case management or a combination of support (see Chapter 3 and Chapter 4). Operational details may also need to vary to allow for differences in context between European countries, e.g. differences in health, welfare and housing systems. However, adherence to the core principles of Housing First cannot vary if a service is to be viewed as high fidelity.
Examples of Housing First fidelity tests include:
- The Pathways to Housing First fidelity measure((http://www.housingfirsttoolkit.ca/sites/default/files/Revised_HF_Self-Assessment_Survey_12-23-13.pdf)).
- The Canadian At Home/Chez Soi programme fidelity measure((http://www.housingfirsttoolkit.ca/sites/default/files/AtHomeFidelityScale.pdf)).
- The Full Service Partnership (FSP) fidelity measure((http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4097835)).
5.2.3. Effectiveness Evaluation
The evaluation of effectiveness centres on what a Housing First service is achieving. This aspect of evaluation includes the progress that a Housing First service is making in terms of delivering the outcomes it is designed to deliver. It is also important for an evaluation to understand what the people using a Housing First service think about Housing First.
The evaluation of effectiveness starts by exploring the ways in which a Housing First service is delivered. Alongside understanding the structure of the Housing First service and observing how it works, this also involves mapping the range of partner agencies involved, how the service is funded and how the networks that a Housing First service relies on are structured and function. In order to understand the effectiveness of a Housing First service, it is very important to understand how the Housing First service is designed and how it operates. This involves understanding how a Housing First service is targeted, what it is designed to achieve and what the roles of the Housing First staff team are.
After assessing fidelity, an evaluation must explore the outcomes that a Housing First service is designed to achieve. This means testing whether or not a Housing First service is achieving what it is supposed to achieve, both in terms of outcomes and the views of the people using the Housing First service.
Exploring the effectiveness of a Housing First service has several dimensions:
- Promoting housing sustainment and a lasting exit from homelessness. This is sometimes also called housing retention.
- Enhancing the health and well-being of Housing First service users,
- Improving the social integration of Housing First service users.
- The cost-effectiveness of Housing First.
