MAE logo Welcome to MAE!

To submit articles for the newsletter, e-mail the secretariat at info@maeeval.org

Summer 2008 NEWSLETTER

The President’s Message

by Meg Blinkiewicz

Happy Belated Summer Solstice, MAE members! As we enjoy these summer months in our great state, MAE is taking the time to reflect—to build on our successes and shore up areas that need improvement. As has become our tradition, the board completed its annual planning retreat in early June. I’d like to take this opportunity to share some of what was discussed at this important meeting.

First, the board has pledged to continue its efforts to build strategic partnerships with organizations that share common elements of our mission. We hope that by taking the time to systematically build these partnerships, we enhance MAE’s exposure among funders and users of evaluation, stabilize our financial position, improve our understanding of our constituents’ needs, generate focused professional development opportunities for evaluators and practitioners, and improve the practice of professional evaluators.

Second, look for information about our 2008–09 professional development offerings in the early fall, along with preliminary information regarding the 2009 conference. The board is attempting to create a menu of workshops that meets the needs of our three constituent groups: evaluators, practitioners, and funders. We also are attempting to offer these trainings in different geographic regions of the state. Look for our outreach and communication vehicles—the website and newsletter—to be filled with useful information about new evaluation techniques, trainings, requests for proposals, book reviews, and exciting MAE member news.

As always, we are looking for MAE members to step up and assist board members with our work. If you have ideas and energy, please consider applying them to MAE! We can meet our goals only through the combined efforts of our board and membership. You can reach me directly at blinkiewicz@earthlink.net with comments, ideas, and questions.

Meg V. Blinkiewicz
President

Unlocking the Black Box of Collaboration

By Ruth A. Mohr, Mohr Program Evaluation & Planning, LLC
The following is based on a presentation given at the May 2008 Society for Public Health Education (SOPHE) Midyear Scientific Conference in Chicago.

We live in a time when no organization can succeed on its own…As we look around us in a new century, we realize that businesses and non-profits in today’s interconnected world will neither thrive nor survive with visions confined within the walls of their own organizations. They need to look beyond walls and find partners who can help achieve greater results and build the vital communities to meet challenges ahead. (Hesselbein and Whitehead 2000)

Collaboration is an imperative for many health programs, nonprofit organizations, governmental agencies, educational institutions, and communities. By working together, communities and organizations hope to pool scarce resources (talent and time as well as money) and achieve a shared vision that would not be possible working independently. Evaluators are more and more frequently asked to assess programs that have a collaborative component.

However, there is little agreement, often even within programs, as to what collaboration means. The term is often used generically without careful thought being given to what level of working together is needed to achieve a desired outcome or what type of preparation participants may need to work together successfully. It appears a “black box.”

Evaluators can help programs define working together more clearly so that the role of collaboration in achieving the program’s goals can be explored more successfully. Three early steps can help set the stage for effective working together and provide a foundation for developing and timing evaluation. These are: treating collaboration as an intervention in its own right and consciously including its role in achieving the shared goal in the Theory of Change (TOC); developing literacy about collaboration and a common collaboration language among important stakeholders; and assessing important contextual factors for their effect on the ability of the desired level of working together to be achieved.

These steps are interactive. Part of determining what must happen for the program to achieve its goal is determining what level of working together is necessary. This gets addressed when the TOC is developed. Part of determining what level of working together is possible is determined when assessing the context in which it will occur. An ongoing challenge is assuring that those participating in the collaborative effort both agree on how they need to work together and on what words they will use to communicate with one another about what they are doing.

Model

Collaboration as Part of the Theory of Change

Requiring collaboration without careful thought being given to what the collaborative intervention is expected to add and what it needs to look like can place under- or over- expectations on participants. Different levels of working together (e.g., networking/sharing information, cooperation, coordination, joint strategic interdependent action) are appropriate for achieving different goals. The term “collaboration” can mean any and all of these so the specific meaning needs to be included in the TOC.

For example, developing a high or intense level of working together is costly in terms of time and effort needed to institute and maintain the process as well as in terms of the education and training required to develop appropriate skills in the participants to do it successfully. Therefore, it doesn’t make sense to use a resource-intensive, strategic, interdependent action approach to working together if all that is needed is to share information or coordinate services. On the other hand, one shouldn’t expect to change a long-standing system with an informal exchange of ideas.

To make things a bit more complex to think about, a program or initiative may use different types of working relationships at different times to achieve sub-goals that are part of a larger goal. And, as environmental conditions change and more is learned about what is needed to change complex issues, the ways that partners need to work together can also change. In this latter case, the TOC needs to be reviewed and revised as needed. Overall, the goal is to make the level of working together “fit” the end desired. Specifying the role of collaboration helps keep everyone focused on what differences they are trying to make and which methods and strategies they want to use. It also helps clarify what is appropriate to evaluate, and when.

Collaboration Literacy and Common Language

The term “collaboration” persists as an abstract and normative descriptor. When organizations say that they want to work “collaboratively,” each one may have a different mental model of what that means above and beyond whether they have actually considered what type of working together would be required to achieve their shared goal.

Gajda and Koliba (2007), two evaluators of collaboration in education, recommend helping participants understand conceptually how collaboration is different from how they might be used to working together and semantically how they want to describe what they are doing in their specific situation since collaboration is often used to describe a wide variety of behaviors (e.g., networking, coordination, partnering).

Assessing Contextual Factors

Context has an impact on the expectations of various participants in inter-organizational work and on how the act of collaboration plays out. A partial list of contextual factors to be aware of and take into account is included here:

What is learned from modeling the Theory of Change and from the assessment of contextual factors helps to identify the skills required to manage both the level of collaboration desired and the potential impact of contextual factors. Mandell and Steelman (2003) call for managers to be aware that managing collaboratives is different from managing traditional organizations and emphasizes potentially different skills (e.g., trust building, creating environments that support productive interaction and communication). They note the strong role that contextual factors can play in thwarting the best laid plans and intentions and that managers need to be prepared to strategically address them.

In summary, to start unlocking the “black box of collaboration,” it can be useful to

Doing the above can improve the “fit” of collaborative approaches with program or initiative needs and provide a strong foundation for developing and timing evaluation.

References

Gajda, R. & Koliba, C. (2007). Evaluating the imperative of intraorganizational collaboration: A school improvement perspective. American Journal of Evaluation 28:1 (26–44).

Hesselbein, F. & Whitehead, J. (2000). Foreword to J. Austin, The collaboration challenge: How non-profits and businesses succeed through strategic alliances. San Francisco: Jossey-Bass.

Mandell, M. P. & Steelman, T. A. (2003). Understanding what can be accomplished through interorganizational innovations. Public Management Review 5:2 (197–224).

Call for Articles

The MAE newsletter includes articles on evaluation theory, practice, and methods, as well as articles on related technology and tools, to assist evaluators, funders, and program stakeholders in their work. Articles should be original works or previously published with documentation of permission to reprint, 500 to 1,500 words in length, and submitted electronically. Articles may be submitted at any time for consideration by MAE; the person submitting the article will be notified prior to its publication.

Please submit your articles to the MAE Secretariat at info@maeeval.org.

Professional Development Opportunities

American Evaluation Association Annual Conference: Evaluation Policy and Evaluation Practice

The American Evaluation Association's annual conference, Evaluation Policy and Evaluation Practice, is scheduled for November 5-8, 2008 in Denver, Colorado. Click here for more information and to register.

Evaluation Resources: Check it out Check it out

The Evaluation Center at Western Michigan University’s website has many resources for evaluation, including publications and links. Visit the website by clicking here.

One interesting item to note is the Evaluation Café. The Center faculty and graduate students hold one-hour presentations on a variety of evaluation topics a few times each month. It is not a problem if you are not near Western—the website posts the talks in streaming video form, as well as slides for many presentations. Recent topics include Evaluating the Kalamazoo Promise; RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints; and Concepts Underlying Evaluating Organizational Effectiveness.

Look for more Professional Development Opportunities in the FALL of 2008!

Tentative topics include: evaluation essential skills training, quality assurance for evaluation businesses, and SPSS and/or SPSS syntax.

Have a group that needs to know about evaluation?

The MAE can provide a learning opportunity tailored to your organization’s needs! Contact MAE Professional Development Committee Co-Chairs Elizabeth Agius (eagius@wayne.edu) or Neva Nahan (n.nahan@wayne.edu) for more information.

MAE Member News

New MAE board member, Ruth Mohr of Mohr Program Evaluation & Planning, LLC, presented “Unlocking the Black Box of Collaboration: Step 1” at the SOPHE Midyear Scientific Conference in Chicago on May 23, 2008.

Chris Coryn, Ph.D, Director of the Interdisciplinary Ph.D in Evaluation at Western Michigan University, has recently published the article, “The Logic of Research Evaluation,” in New Directions for Evaluation, No. 117, fall of 2007, a publication of Jossey-Bass and the American Evaluation Association. This fall Dr. Coryn will teach Foundations of Evaluation (Eval 6000) and Interdisciplinary Seminar in Evaluation (Eval 6010) at Western Michigan University.

Enroll Now!
Click here to view the MAE Membership brochure.

Become a member today!

MAE has recently updated its annual membership to a rolling enrollment. This means your membership is now valid for 12 months no matter when you enroll! You can renew your enrollment or become a member for the first time with the MAE online by clicking here.

Submit your news!

As a service to our members, the MAE is adding a Member News section to our newsletter. Although this section is viewed by our larger audience, only current MAE members may post information. If you want to share the news of a recent promotion, project, achievement, training, contract, or any other professional milestone, please send the information listed below to the MAE Secretariat at info@maeeval.org in order to be included in upcoming newsletters.

This Newsletter is Sponsored By

MSU’s Community Evaluation and Research Center (CERC) acts as a hub for program evaluation activity across Michigan State University.

CERC provides training in evaluation and community-based participatory research, and conducts formative and summative evaluations.

CERC addresses complex human, organizational, and social issues using a systems, developmental, and participatory approach.

Find more information at
outreach.msu.edu/cerc

MAE Secretariat Contact Information

Jacqueline LaFay
MAE Secretariat
c/o Public Sector Consultants
600 W Saint Joseph Street, Suite 10
Lansing, MI 48933
Phone: (517) 484-4954
Fax: (517) 484-6549
E-mail: jlafay@pscinc.com

Newsletter Feedback

Please let us know how you like (or dislike) the new format of the MAE Newsletter. Please e-mail your comments to info@maeeval.org.