MAE logo Welcome to MAE!

To submit articles for the newsletter, e-mail the secretariat at info@maeeval.org

Early Spring 2008 NEWSLETTER

The President’s Message

by Meg Blinkiewicz

Over the years I've come to associate many things with spring, including our annual MAE conference. Looking out my window at yet another snowy Michigan day, I realize I'm in need of a day of learning and sharing more than ever. Our 2008 conference promises to meet the needs of all of our members through an intriguing keynote address and a wide range of high-quality workshops.

Plan to join us on Monday, March 31, in Lansing for the 13th annual conference, Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs. Our keynote speaker, Dr. Melvin Mark, will share his framework for understanding how programs and policies contribute to social betterment. The six afternoon workshops provide participants with a wide variety of learning opportunities. Finally, we'll have an informal networking time where I hope to meet and chat with many MAE members and friends. We hope to see more than 100 participants at the conference so please invite your colleagues to attend.

In addition to the Monday events, please consider joining me and other MAE members for informal networking with Melvin Mark on Sunday evening. We'll plan to convene in the early evening to chat with Dr. Mark about MAE, evaluation, and our lovely spring weather!

I'm looking forward to seeing each of you on the 31st in Lansing.

Creating the Capacity to Continously Improve a System of Care

by Suzanne Miel-Uken, MS, Impact Lead Evaluator; Vice President for Health and Human Services, Public Sector Consultants Inc.; Matt Wojack, Impact Project Director, Clinton-Eaton-Ingham Community Mental Health; Malisa Pearson, Impact Lead Family Contact, Association for Children's Mental Health.

Introduction

Closing the gap that often exists between managing an initiative and measuring results takes getting beyond asking "What works?" to asking "What do we need to know to improve?" The Impact System of Care, a partnership of child- and family-serving agencies working in cooperation to provide coordinated services for children with serious emotional disturbance (SED) and their families, is closing that gap by integrating evaluation and performance measurement, with families, youth, service providers, and administrators as key decision makers in performance improvement. System ownership is expanding and data is used quickly to refine strategies and activities to strengthen a system of care.

Building a system of care for children with severe emotional disturbance and their families is a complex endeavor that requires recurring activities to monitor progress toward goals and to make adjustments that will enhance system performance. These activities are intrinsic to what managers must do in organizations, and do best when relying on the methods and tools of evaluation to monitor and strengthen performance. However, in a system of care, agreeing on goals and monitoring progress to reach those goals must go beyond managers, methods, and tools. To move toward a system of care that is increasingly family-driven, youth-guided, and culturally competent, monitoring goal achievement and making adjustments to accelerate progress must be done through relationships among all stakeholders, with families and youth as full partners.

The full engagement of all stakeholders in system building and improvement is dependent on a range of capacities and interactions. The Impact System of Care is developing among all stakeholders the capacity to determine the data needed to manage and measure performance, use that data to generate information and knowledge for improving the developing system of care, and monitor progress toward outcomes. Just as importantly, Impact has established a set of interactions and activities among the stakeholders to participate in the key steps in the continuing cycle of action, assessment, and refinement. The cycle spins quickly, looping through all steps every three months.

Underlying the capacities, interactions, and activities are essential relationships of trust that are being nurtured and deepened among the stakeholders. The underpinning of trusting and candid relationships among stakeholders, particularly between families and youth and other system of care partners, is what makes it possible to develop the capacities needed to address weaknesses and build on strengths, thus, ever improving the system of care.

The Cycle of Continuous Quality Improvement

The web of activities and connections among the system of care stakeholders is best described by the cycling that completes each round of quarterly performance, evaluation, and improvement. Key steps in the quarterly cycle are depicted in Figure 1, which shows how the interactions for the purpose of evaluation and performance management—and more importantly, for performance improvement—are fully integrated. The responsible participants in each key activity are noted.

Figure 1

Impact cycle

How did Impact create its continuous cycle of improvement?

What does it take to make a knowledge management process like this work?

What is Impact's next step?

During the second year of implementation, Impact is honing in on the right mix of indicators, with an emphasis on outcome measurement, while continuously improving performance based on results and what the stakeholders believe they need to know in order to continuously improve the system of care. Another priority is strengthening youth involvement in the evaluation and improvement of the system of care.

Call for Articles

The MAE newsletter includes articles on evaluation theory, practice, and methods, as well as articles on related technology and tools, to assist evaluators, funders, and program stakeholders in their work. Articles should be original works or previously published with documentation of permission to reprint, 500 to 1,500 words in length, and submitted electronically. Articles may be submitted at any time for consideration by MAE; the person submitting the article will be notified prior to its publication.

Please submit your articles to the MAE Secretariat at info@maeeval.org.

2008 Conference Registration

MAE 13th Annual Conference

Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs

Featuring Dr. Melvin Mark of Penn State University, editor emeritus of the American Journal of Evaluation, and past president of the American Evaluation Association, as the keynote speaker.

MARCH 31, 2008

Lansing, Michigan, at the Radisson Downtown

Who should attend?

Anyone involved with or interested in evaluation, including funders, administrators, program managers, students, quality control or evaluation staff, and consultants will want to attend.

After his keynote presentation, Dr. Mark will entertain questions from the audience and facilitate a discussion. Following a complimentary lunch, further opportunities for learning and discussion are scheduled. Adhering to the mission of the MAE, a variety of workshops are planned to meet the needs of persons with varying degrees of evaluation knowledge and experience.

Christian Coryn, Evaluation Center at Western Michigan University, will present The Field of Evaluation 101. This interactive session is designed to orient those new to the field.

Knowledge Management 101, presented by Cynthia Phillips, Phillips Wyatt Knowlton Inc., will provide an introduction to knowledge management and demonstrate its strategic application to evaluation. Often overwhelmed by data, those conducting program evaluations will appreciate this approach, which helps them distill what’s important and apply this new knowledge to improve programs and policies.

Principled Discovery: Learning from your data, with reason and What Works When: Unraveling how context affects program effectiveness are two sessions geared to the more experienced evaluator and presented by the keynote speaker, Dr. Mark.

An additional session called The Success Case Method: Measuring and accelerating impact of training (and other change initiatives) simply, credibly, and quickly, presented by Robert Brinkerhoff, is also geared to more experienced evaluators.

The Funders’ Perspectives on Evaluation will provide valuable insights to everyone. This panel discussion between funders will address what several Michigan foundations look for in both evaluations and evaluators.

To end the day, please join fellow conference attendees for an informal networking gathering. Snacks and drinks will be available and a variety of items donated by members will be raffled off. Two complimentary raffle tickets are included in your registration packet.

Register by March 3 for early registration rates, and of course, there is a discounted registration price for MAE members. Click here to register online.

  Registration
Members $120
Students $60
Non-Members $160

For more information about the conference program, or if you wish to be a conference sponsor, contact the conference co-chairs: Meg Blinkiewicz at (269) 929-1982 or Liz Agius at (313) 577-5251.

Professional Development Opportunities

Hierarchial Linear Modeling

Presenter: Jessaca Spybrook, PhD, WMU Evaluation Center
Date:   Thursday, May 8, 2008
Time: 1:00–4:00 PM
Place: University of Michigan-Dearborn, Fairlane Center, 19000 Hubbard Dr., Dearborn, MI 48126
Cost: $40 for MAE Members, $60 for Non-members

Do you work with multilevel data? For example, patients nested within hospitals (2 levels of data) or students nested within classrooms, which are nested within schools (3 levels of data). Have you struggled with analyzing and understanding the different levels of data? If so, we have the workshop for you!

HLM is a statistical technique for analyzing hierarchical or nested data that appropriately accounts for the various levels of data. This workshop provides an overview of the usefulness of HLM and a hands-on introduction to the HLM software.

Dr. Jessaca Spybrook has expertise in using HLM as well as teaching first-time users how to understand how HLM works, its benefits, and how to use it on evaluation and research projects. Dr. Spybrook is a senior researcher at The Evaluation Center. Her expertise is in the design and analysis of experiments that test the effectiveness of educational interventions. She coauthored the software and documentation for Optimal Design, a program that assists researchers in planning adequately powered studies. Dr. Spybrook holds a master’s degree in applied statistics and a doctorate in education from the University of Michigan.

Participants are asked to bring a laptop and install the HLM student version ahead of time so that they can participate in the workshop. The HLM student version is free and can be downloaded by clicking here.

Please note: This workshop also will be presented on the east side of the state in spring 2008.

Tools for a Learning Organization-Michigan Nonprofit Association Super Conference Pre-Session

Presenters: Neva Nahan and Ruth Mohr
Date: Monday, May 12, 2008
Time: 1:00–4:00 PM
Place: Lansing Center, 333 E. Michigan Ave., Lansing, MI  48933
Cost: $35

This workshop will provide participants with skills to help their nonprofit organizations further along the road to becoming learning organizations. The key to learning is systematically collected data on questions important to organizational mission. Learn to identify and use data to document successes, identify weaknesses and problems, and measure progress toward goals. Make decisions and allocate resources based on evidence of what is working and what is not. This workshop will include practical approaches, and participants will leave with tools to support a new mindset that embraces data as an essential tool for improvement.

Summer Research Training Institute: Cluster-Randomized Trials

The National Center for Education Research in the Institute of Education Sciences (IES) of the U.S. Department of Education has announced it second Summer Research Training Institute on Cluster-Randomized Trials to increase the national capacity of researchers to develop and conduct rigorous evaluations of the impact of education interventions.

When: July 6–18, 2008
Where: Northwestern University, Evanston, IL
To apply: Information is available on the IES website; click here for further information.

For assistance please contact Dr. Christina Chhin at (202) 219-2280 or christina.chhin@ed.gov.

MAE Member News

Elizabeth Agius, recently moved from the Wayne State University School of Medicine to the newly formed Center for Social Work Practice and Policy Research at WSU School of Social Work. At the Center she serves as the Manager of Community Research Partnerships and facilitates efforts to enhance research through community partnerships. Ms. Agius's evaluation practice and publications include the arenas of schools, nonprofit organizations, substance abuse prevention programs, and capacity building research. She is currently serving the second year of her first MAE Board term.

New Newsletter Feature!

As a service to our members, the MAE is adding a Member News section to our newsletter. Although this section is viewed by our larger audience, only current MAE members may post information. If you want to share the news of a recent promotion, project, achievement, training, contract, or any other professional milestone, please send the information listed below to the MAE Secretariat at info@maeeval.org in order to be included in upcoming newsletters.

MAE Membership

Click here to view the MAE Membership brochure.

MAE Secretariat Contact Information

Jacqueline LaFay
MAE Secretariat
c/o Public Sector Consultants
600 W Saint Joseph Street, Suite 10
Lansing, MI 48933
Phone: (517) 484-4954
Fax: (517) 484-6549
E-mail: jlafay@pscinc.com

Newsletter Feedback

Please let us know how you like (or dislike) the new format of the MAE Newsletter. Please e-mail your comments to info@maeeval.org.