Chapter 4 Program Monitoring

As stewards of public funding, the Serve Illinois Commission needs to ensure the programs they fund are aware of their contractual requirements and are in compliance with all of the rules, regulations, and provisions governing AmeriCorps funds. To accomplish this, the Commission Program Officer must create adequate systems for monitoring programs.

First year programs will receive a site visit by their Program Officer during to the start of their first grant cycle. In addition, first year programs will be considered high risk and will also have a monitoring review scheduled during their first year.

Programs with staff changes, i.e., Program Director, will also be considered high risk and will have a monitoring review scheduled during the first year with the new staff.

Other factors that may determine if your program is at high risk, include, but are not limited to, late reporting, not participating in mandatory calls, trainings, and serious issues that arise during the Program Year, etc.

This approach helps to ensure high quality AmeriCorps programs are implemented while maximizing staff time and meeting the recommendations of the Inspector General in regards to program monitoring.

Based on a state or organization’s risk-based monitoring strategy, a programmatic site visit may be made to every awardee or site one time per grant cycle, one time per year, or more often as may be appropriate. It is the goal of the Commission to complete monitoring visits to each program every year; however, this depends on staffing levels.

In most cases, on-site monitoring visits will be scheduled at least 30 days in advance. Your Program Officer will contact you to schedule the monitoring visit for your program.

To prepare properly for an on-site monitoring visit, your program staff should:

• Make certain appropriate staff will be available.

• Send all documents electronically ahead of time per the instructions you will receive.

• Understand that Program Directors are not included in the Member interviews.

• Review the Documentation request provided by your Program Officer and be prepared to send the requested documentation electronically.

• Make certain all required forms are in the Member files and in the order listed on the Member File Checklist.

• Verify that Member time sheets are up to date and signed/dated by the Member and their immediate supervisor (from their host site).

Typical on-site activities during the Monitoring Review include but not limited to the following:

1. Discussion with Program staff;

2. Member File documentation compliance check;

3. Financial compliance check;

4. Policy & Procedures compliance check;

5. Host site visit; and

6. Member interviews.

The purpose of an on-site monitoring visit is to assess the program’s compliance with federal regulations and to help the program improve systems to pass an OIG audit. Member files, program policies and fiscal records will be reviewed during the visit. The Serve Illinois AmeriCorps Program Review Instrument will be used to assess program compliance.

On-Site visits by program staff can have many purposes, some of the most important are summarized in the table below:

During the on-site monitoring visit, your Program Officer will use the following modules of the Serve Illinois AmeriCorps Program Review Instrument:

Module A: Reporting and Communication Compliance and Early Issue Detection

Portions of this Module will be completed by your Program Officer prior to the monitoring visit using information provided in eGrants. Results will be reviewed with the program and important issues, or concerns will be discussed. Programs that are not performing in this area must show improvement and demonstrate a commitment to improving. Any concerns in this area will be addressed in the site visit feedback letter and a corrective action plan to improve future performance will be required.

Module B: Financial Compliance

The majority of this module will be completed with the assistance of the program’s fiscal staff, but some questions will most likely be answered by program staff. Programs should have copies of their expenditure forms and the Federal Financial Reports submitted via EGrAMS available for review. A copy of the program’s final approved budget should also be available. In general, a program must have all required documentation for each item on the list.

Module C: Policies and Procedures Compliance

All programs must be in compliance with all of the policies and procedures listed in Module C. Documentation and/or proof of compliance is necessary in case of an audit. Programs must have all documentation listed for this compliance check readily accessible to your Program Officer during the monitoring review.

Module D: Member Documentation Compliance

Your Program Officer is required to review ten percent (10%) or ten (10) Member Files, whichever is greater. If your program has 40 Members, 10 Member Files are required to be reviewed. If your program has 300 Members, 30 Member Files are required to be reviewed.

Member files are reviewed to determine if required documentation is being kept by the program. Proof of compliance is necessary in case of an audit. Your Program Officer will randomly select the Member files to be reviewed. All Member files reviewed must contain the required documentation and be placed in order in the Member file as listed on the MODULE D: Member Documentation Compliance form. In addition, each Member file must have the Member File Coversheet along with the completed documents attached to it.

Module D2: Staff or Site Supervisor Compliance

Your Program Officer is required to review all Staff listed under Section 1 of the application. Staff files reviewed must contain the required documentation and be placed in order in the Staff file as listed in MODULE D2.

Module E: Host Site Visit

You and the monitoring team will visit one or more sites where Members are serving.

Module F: Member Interviews

The monitoring team will interview a group of Members during the visit. Program staff will not be involved in this interview, and responses will be kept confidential. A summary of the discussion will be made available with the rest of the monitoring report.

*See Attachments –

• Program Review Instrument – Monitoring Tool
• Member Service Agreement (MSA)
• Member File Coversheet

Review Outcome

An AmeriCorps Program Review will be sent to the Program Director and/or designee by your Program Officer which may request additional documents. Programs will have 10 working days to respond with the requested documentation showing full compliance. Once the review is completed, the Program Officer will notify the Program Director and/or designee in writing that the monitoring review has been completed and all issues have been resolved. Copies of all completed forms and correspondence sent to the SIC Program Officer will be kept in the program file.

Programs that are not performing must show improvement and demonstrate a commitment to improving. Any concerns in this area will be addressed in the Program Review and a corrective action plan to improve future performance will be required.

Programs designated as high risk or who request a site visit to focus on specific issues may receive additional visits.

High risk factors may include, but are not limited to:

• A first-year program

• A change in program director or other key staff

• Legitimate Member complaints to the Commission

• Poor past performance by the program (based on progress reports and/or previous site visits)

• Concerns regarding prohibited activities

• Compliance with reporting deadlines and Financial Compliance Issues

These visits will be conducted on an on-going basis throughout the program year. High Risk on- site visits may not be scheduled 30 days in advance. These visits will occur when issues are brought to the attention of the Program Officer. Commission staff will determine the need for a High-risk on-site visit.

High risk on-site visits are conducted to provide targeted assistance on specific issues or concerns.

First year programs will receive an on-site visit by their Program Officer and/or SIC Staff prior to the start of their first grant cycle. In addition, first year programs will be considered high risk and will also have a monitoring review scheduled during their first year.

Programs with key staff changes will also be considered high risk and will have an on-site monitoring review scheduled during the first year with the new staff.

Other factors that may determine if your program is at high risk, include, but are not limited to, late reporting, not participating in mandatory calls, trainings, serious issues that arise during the program year, etc.

For program-related visits being conducted because of high risk factors, the agenda will be determined by the reason for the visit. For example, the agenda for a visit that is being held because of a change in program director will focus on expectations of programs, AmeriCorps regulations and relationship building. The agenda for a visit that is being conducted because of concerns regarding prohibited activities will focus on meeting with Members, site supervisors, and program staff to assess whether prohibited activities were taking place and to ensure that prohibited activities do not occur in the future.

A Program Review will be completed for each High-Risk On-Site Visit.

While an on-site monitoring visit can provide a valuable, direct view of a program’s operations, it is a resource-intense form of monitoring. Desk-based monitoring is more sparing of scarce resources of staff time and money and, properly used, can forestall the need for more intensive intervention later on to resolve awardee problems.

Throughout the year, your Program Officer manages each program in their portfolios. Areas that require your Program Officers attention include:

• Reviewing and approving performance measures

• Reviewing timeliness of reporting

• Reviewing program participation in required calls, trainings, service days & Recognition Day

• Reviewing each program/site’s progress on Member recruitment and retention

• Tracking Member progress toward completing service hours

• Conducting Member enrollment queries

• Review progress towards performance measures

Your Program Officer will also utilize PPRS AND EDFS as effective management and monitoring tools.

The Member File Cover Sheet and Checklist are required forms that are to be used in all Member files by all programs. These mandatory forms must be placed at the top or the beginning of the Member file. These forms not only serve as a checklist for all the items that are required to be in the Member file, but it also serves a tool to help you keep all the items in the proper order and sequence. It is a working sheet that will be updated and completed during the course of the Member’s service and will serve as a quick reference on nearly all aspects of Member information. It is also a valuable tool to help you with retrieval of information and will be helpful to the Commission during reviews and monitoring of Member files.

*See Attachment - Member File Coversheet and Checklist

Member Service Agreement

As stated in Chapter 3, the Member Service Agreement (MSA) is a critical part of administering AmeriCorps programs. The Member Service Agreement provides the legal basis or contract by which the terms, conditions and rules regarding participation are delineated. This contract between the program and Member is the legal document which would be used to resolve any issues between the program and a Member. Because of its importance, The Commission has developed a template for the programs to use. The template includes all the required provisions required by the AmeriCorps Agency. Programs are required to use the Serve Illinois MSA for each Member. You can place your logo on the MSA, but you may not alter it and it must demonstrate to be the original Serve Illinois MSA.

Programs will need to complete various parts of the Agreement to make it specific to the Member’s service. Programs will also need to attach/insert the position description to the Agreement. Signed and completed Member Service Agreements are required to be in each Members file.

Member Orientation

Orientation materials should also be reviewed carefully to ensure that your program has provided Members with appropriate training for beginning their term of service. We have provided you with a SIC AmeriCorps Member Orientation (required items) Checklist. Please use this checklist to make certain all required topics are covered during your Orientation.

In light of the AMERICORPS NATIONAL standards for Member enrollment and retention, it is important for your Program Officer to monitor their awardees/sites closely to keep track of how they are doing in these areas. The standards are:

ENROLLMENT RATE = Total slots filled/total slots awarded.

Enrollment for programs should be 100%.

RETENTION RATE = Total Members exiting with an Education Award/total slots filled.

A target of 90% for retention or completion is desired.

AMERICORPS NATIONAL realizes that many programs not yet meeting these rates are nevertheless performing well in their individual circumstances. Also, retention may vary among effective programs depending on the target populations for those programs. However, in order to leverage limited program dollars, AMERICORPS NATIONAL expects programs to strive toward improving their rates of enrollment and retention. During the application process, programs that are not meeting these standards need to offer a rationale for why not, as appropriate.

Recruitment/Retention Rate Calculation

To calculate enrollment and retention using the data in My AmeriCorps:

1. Log into your eGrants account

2. Select S/N Reports

3. Select Report via pull down menu

4. Select Program year

5. Select format (PDF, Excel etc.)

6. Select Submit

7. You may print the report created

The AmeriCorps Agency grants are usually awarded for a three-year Project Period, consisting of three yearly budget periods. The Serve Illinois Commission is required to submit documentation of program close outs to the AmeriCorps Agency.

The following Final Close-Out reports are due to your Program Officer within 60 days after the close of your 3-year grant cycle. (Close out begins once all of your Members have completed their term of service.)

Final Project Report

The narrative should include a three-page, double-spaced summary of the qualitative accomplishments that the AmeriCorps grant has made for the duration of the grant period. Include a discussion of the impacts of any special initiatives that fall within this grant. Discuss what exists now in the communities that are served that did not exist prior to this grant. Quantitative data may be used to support the impact statements.

Equipment and Supply Inventory Form

A program completing the final year of its three-year grant cycle must submit two (2) inventory forms. One for any equipment inventory with a current fair market value of $5,000 or more purchased with federal grant funds. The second form for unused/residual supplies inventory purchased with federal funds exceeding $5000. Reports should be completed on the required forms. If no equipment or supplies were purchased, forms should reflect such. For more information, please refer to the AmeriCorps Provisions.

IDHS Grant Close-Out

At the end of each IDHS grant period, providers will be required to complete a certain documents. Your program officer will provide you with any documents for close out that are required by IDHS.

*See Attachments -

• Closeout Equipment Inventory
• Closeout Inventory of Residual Supplies
• Certification of Subgrant Closeout

As articulated in the AmeriCorps regulations 45 CFR §§ 2522.700-740, all AmeriCorps State and National awardees that receive an average annual AmeriCorps Agency grant of $500,000 or more must conduct an independent evaluation. An independent evaluation uses an external evaluator who has no formal or personal relationship with, or stake in the administration, management, or finances of the awardee or of the program to be evaluated.

All other AmeriCorps State and National awardees must conduct an internal evaluation. An internal evaluation is designed and conducted by qualified program staff or other stakeholders, such as board Members, partners, or volunteer affiliates.

If you are recompeting for the first time, you are required to submit “a summary of your evaluation efforts or plan to date, and a copy of any evaluation that has been completed, as part of your application for funding” (45 CFR § 2522.730). If you are recompeting again, you are required to submit a completed evaluation with your application. The AmeriCorps Agency will consider the results of your evaluation “in assessing the quality and outcomes of your program” (45 CFR § 2522.470).

If you receive an average of $500,000 or more per year from the AmeriCorps Agency, averaged over the last three years of funding you have received before you recompete, we expect you to conduct an independent evaluation by contracting with an external evaluator. The AmeriCorps regulations describe how this evaluation should provide evidence of a causal relationship between program activities and outcomes (45 CFR § 2522.700). You may consider using an experimental or quasi- experimental design or compare your results with national/state/local data. Your external evaluation method should match the size, scale, and purpose of your program.

In our ongoing effort to reduce burden on awardees, especially those with smaller grants, awardees an average annual grant under $500,000 may submit an internal evaluation. The primary difference between the independent evaluations that awardees that receive $500,000 or over are required to submit and the internal evaluation is who conducts the evaluation study. Your own staff and other stakeholders can serve as internal evaluators.

We encourage you to design your internal evaluation so that it will yield data most useful to you. You may opt for an impact evaluation, or you may conduct a process or management evaluation. You are not required to conduct an experimental or quasi-experimental evaluation that proves causality, which is required of awardees that receive $500,000 or over, although you are allowed to conduct this type of study. We expect the same high quality that we expect of a larger awardee, regardless of the type of evaluation you decide to conduct.