What is Evaluation Policy and Why Does it Matter?
Welcome to the April 2023 edition of the Improved Insights Newsletter: a newsletter for folks in informal STEM learning spaces who want to learn more about how to measure and communicate the impacts of their work.
I’m Sarah M. Dunifon - a long-time STEM educator, evaluator, and equity advocate. After spending time working in museums, nonprofits, and higher education, I now work on cutting-edge informal STEM learning research and evaluation with my team at Improved Insights. We specialize in informal STEM learning and youth programs.
This monthly newsletter includes tips, resources, and news on informal STEM learning evaluation. We hope you find it valuable! Now, let’s get started —
What is Evaluation Policy and Why Does it Matter?
In this month's newsletter, we'll talk about evaluation policy, what it is, and why it should matter to you! To read the full article, check it out in our Insights. In the meantime, here’s a taste:
While it might seem like a rather dry subject area with little actual influence on your work, evaluation policy is enormously important for informal STEM education (ISE) organizations. And not just in the sense of “you should develop your own policies” (and yes, maybe you should), but rather, evaluation policy is a commitment to certain values. Values of the organization, values of the individuals with power in those organizations, values that communicate and uphold certain standards in not just the field of evaluation, but ISE as a whole.
Let’s talk about how these values can present themselves, using two super fancy words that your coworkers will be impressed to hear you say: axiology and epistemology.
Axiology is the study of values and the intrinsic worth of things. It can include ideas like the priorities and importance of beliefs. In the space of ISE evaluation, this might look like: setting priorities for the outcomes you’d like to track (e.g., workforce development over social-emotional learning) or the evaluation questions to include in the evaluation plan.
Epistemology is the study of knowledge and beliefs. It can include ideas like what constitutes knowledge, what knowledge is valuable, and how it can be acquired. In ISE evaluation, this might look like: valuing certain knowledge over others (e.g., “rigorous” quantitative research over Indigenous knowledge), or who is allowed to carry out an evaluation.
You might be wondering why I put the word “rigorous” in quotations in the last paragraph. Well, concepts like rigor, outcomes, and what constitutes the “value” of a program may vary drastically between stakeholders, and are culturally informed. Some have even argued that rigorous requirements (including evaluation) set by foundation staff serve as barriers that perpetuate inequity among nonprofits and the communities they serve (Villanueva, 2018).
So, it is clear that evaluation policy, which conveys values and impacts practice, has real and important implications for our field. The first step to understanding evaluation policy and to improving it in service of equity and better outcomes for all is to be able to identify where policy exists, both explicitly and implicitly, and to ensure those values are communicated clearly.
Since much of evaluation policy is implicit (meaning, it’s “known” within the organization but not written down or communicated outwardly), a great step at your own organization might be to get really clear about what your evaluation policies are, and what they convey about your organization’s values. Then, make them clear to others.
Read more about evaluation policy on our Insights blog
ii Updates
The Improved Insights team is growing!
We’d like to warmly welcome Dr. Oseela Thomas (Research and Evaluation Consultant), Kirinne Slaughter (Research and Evaluation Consultant), and Emily Neill (Administrative Assistant) to our team!
You can learn more about our awesome consultants on our website. Here are some brief introductions:
Dr. Oseela Thomas has content area expertise in STEM, academic achievement, and positive youth development. Her forte is evaluating K-12 out-of-school time programs and working on projects relevant to traditionally underrepresented and disadvantaged groups.
Kirinne Slaughter is a passionate student of Culturally Responsive and Equitable Evaluation. Most recently, Kirinne focused on evaluating effective empathy interpretive practices and building the evaluation capacity of organizational partners. Her research interests include conservation behavior change, empathy for wildlife, connection to nature, and informal STEM education and evaluation.
Emily Neill supports the team with various administrative, research, and evaluation-related tasks. Alongside her work with Improved Insights, Emily supports other STEM learning organizations in an administrative capacity.
60-Second Suggestions
Here are a few of my favorite things this month, usually pertaining to informal STEM education and evaluation, but occasionally some fun personal stuff, too.
Frontiers for Young Minds provides a collection of freely available scientific articles by distinguished scientists that are shaped for younger audiences by the input of their own young peers. A part of the broader Frontiers family of journals, Frontiers for Young Minds creates content both by and for kids.
The Smithsonian Science Education Center’s Zero Barriers in STEM Education Accessibility and Inclusion Program is an initiative focused on increasing the prevalence of accessible and inclusive practices in STEM education and school culture for all students along the continuum of human ability. The program convenes teams of educators for an education summit in Washington, D.C. to develop logic models that focus on a problem of practice related to accessible and inclusive STEM programs and school culture within their context for students with disabilities. Team and mentor applications are due May 19, 2023 at 5pm Eastern.
The National Oceanic and Atmospheric Administration (NOAA) provides a wealth of volunteer citizen science opportunities, such as recording rainfall, analyzing underwater fish photographs, and storm spotting to name a few. Take a look at current opportunities as we close out Citizen Science Month!
As you move through the week, think about how you use data and evaluation in your own personal life. A fun exercise you might try is looking at the ways you evaluate the health and success of yourself and your household. What are your personal and family values, and how do they affect the ways you approach daily living? You may find that you are exercising your own brand of evaluation policy in some surprising ways!
Until next time - thanks!