This is the first in a series of articles that discuss the importance of measuring the effectiveness of fire prevention and public education programs. This process of evaluation will provide the stakeholders with information regarding a program’s (or project’s) effectiveness. This first article will discuss the reasons we incorporate evaluation as part of our programs and the resources available in the area of program evaluation. At the end of the article, the reader will be able to identify the need to incorporate program evaluation and available resources.
All aspects of fire service deliveries should be evaluated: emergency response, emergency medical services, specialty operations and fire prevention. We will look at the application of evaluation to fire prevention programs, specifically in the areas of fire inspection, code compliance, fire and life safety education, fire plan review and fire investigation. These programs have the greatest impact on community risk reduction because they are identified as preventative rather than reactive.
Evaluation has been a part of many industries over the past 50 years. Private industry, primarily in the business sector, started this practice in the late 1940s and was at its height in the 1980s with the introduction of Management by Objectives (MBO). This was the time that the healthcare industry began to incorporate evaluation into all phases of prevention program development. This was started by large hospital groups to answer questions by funding sources, especially business executives who were looking to qualify and quantify program results. It was then adopted by law enforcement in the 1990s for similar reasons.
The concept was presented to the fire service at the same time through benchmarking and initial performance measurements, but may not have been widely accepted or sought after because of limited funding sources outside of local government. However, at the time, the International City/County Management Association incorporated this information into their book, Managing the Fire Service. This book was developed not only for city managers, but also for fire service managers to assist with management decision-making.
The concept was furthered by the incorporation of program evaluation as part of the accreditation process for fire departments by the Center for Public Service Excellence in 2000. Currently, the Insurance Service Organization has incorporated evaluation of prevention and education programs as a component of their rating process for the fire department. Lastly, the Department of Homeland Security uses program evaluation as a component of any funding of prevention and education programs through the Assistance to Firefighters Grant (AFG).
So, why do we spend the time to evaluate our programs? The program manager needs to do so to show not only our command staff the impact of the program to the community, but also to provide information as feedback to the personnel involved in presentation. We also do this as the command staff to make the budget decisions; yes those tough decisions as to what gets funded and for how much. We are held accountable to funding sources, whether private or public, and are expected to prove the benefit to the community. We must pass this information onto the community, through our Police and Fire Commission and elected officials. The community needs to know what resources are available and the impact of these programs on the reduction of risk of injury, death and property loss.
The “why” will be discussed further in future articles through specific examples. The fire service needs to understand that evaluation will become a large part of the fire service in the coming years and we need to learn more about this area. While it is not expected that we embrace this area in its entirety initially, we must recognize its need to be incorporated a little at a time. The good thing is that we already incorporate some measure of evaluation through monthly or annual run statistics.