December 1, 2014
Hello and welcome to the first post of the Spectrum Center Data Blog, updated on a semi-regular basis by our Evaluation & Assessment Graduate Coordinator, Jean Hardy. This is a space dedicated to presenting tangible results from our office's educational programs. The structure of these posts will generally consist of an introduction, a low-level & more visual overview for those who may not be interested in the nitty-gritty statistics, followed by a higher level view that goes over the results of our hypothesis testing. If you would like to see the raw data for any of our surveys or have any questions, please contact Jean.
This first post will be looking at the surveys surrounding our Ally Development Training done for a group of UM Library staff on October 9th, 2014. For those of you who are not familiar with our programming, the Spectrum Center's LGBTQ Ally Development Training Program, started in 2005, seeks to empower members of the University community to be active social justice allies. More than just a "Safe Zone" training, the Ally Development Training uses a social justice framework to guide participants through a curriculum that bridges knowledge of developmental theory to the lived experiences of LGBTQ and Ally-identified people. Topics include social identities, ally development models, LGBTQ terminology, active bystander intervention strategies, and practice utilizing the day's knowledge and skills in real-life scenarios. This specific training was an abbreviated training, one in which we condense our typical 8-hour curriculum into a more manageable 3-hour training for a university office or department.
Through active engagement in the training, participants will grow in their personal awareness, knowledge, skills, and actions as it relates to their engagement in doing ally work. The purpose of having the Ally Development Training is to promote a campus community in which everyone is treated with respect and dignity. This blog intends to “prove” through quantitative analysis that we are meeting these goals.
Overview
We had a response rate of 28 for our pre-survey, a rate of 22 for our post-survey, with 18 people completing both the pre and the post survey. Our pre-survey asked participants to identify with eight required statements on a Likert Scale (1-5) that were meant to unearth the kinds of previous knowledge and abilities the participants had in LGBTQ terminology, bystander intervention, and ally development. This survey was sent out on October 6th and received 28 responses in the three days leading up to the training.
Our post-survey asked participants to identify with seven of the previous eight required statements, seven statements that pertained to the organization and execution of the training, and three questions about their own identities. One of the eight previous statements had its wording change in an attempt to better reflect the learning experience in the training itself, but in retrospect, the same wording should have been used to allow for a better comparison. That statement has not been included in our pre & post analysis because of the change in wording.
Each statement analyzed in both pre and post surveys registered positive change, statistically significant in all cases. This significance is reviewed more thoroughly in the section that follows this overview. To us, this signifies that our trainings are having the desired effects on our participants. For this section, let’s look at two statements that I deem to be especially reflective and pertinent for the changes we desire to see in our participants:
- I am able to describe at least one area where I can grow in my ally behavior (referred to moving forward as S1)
- I understand the role that my social identities play in the different ways I can engage in ally behavior (referred to moving forward as S2)
These statements address desired outcomes in both the ability for participants to reflect on future changes they can make to be better allies and to reflect on the role(s) that their identities play in engaging in ally behavior.
The positive shift in both proportion and number of respondents being able to identify where they can grow in their ally behavior (despite there being fewer responses in the post-survey) show us that this training has succeeded. 71.4% of respondents in the pre compared to 95.5% in the post agreed with S1.
We see a similar (but more dramatic) shift happening in S2, where 39.3% agreed in the pre and 90.9% agreed in the post.
The Nitty-Gritty
|
(with n=18, df=17, t* of 1.740 tested at 95% or 2.567 at 99% significance level with single-tailed two-sample t test with unequal variances) |
Pre mean (SD) |
Post mean (SD) |
Mean |
t-value |
p-value |
|
S1 – I am able to describe at least one area where I can grow in my ally behavior |
3.72 (1.27) |
4.78 (0.43) |
1.06 |
3.3313 |
.002 |
|
S2 - I understand the role that my social identities play in the different ways I can engage in ally behavior |
3.11 (1.08) |
4.5 (0.71) |
1.39 |
4.5688 |
.001 |
|
S3 – I can describe at least one of my social identities within a social justice context |
3 (1.37) |
4.39 (0.83) |
1.39 |
3.5714 |
.001 |
|
S4 – I can identify multiple ways to engage in ally behavior |
2.67 (1.08) |
4.5 (0.71) |
1.83 |
6.0073 |
.001 |
|
S5 – I am knowledgeable about LGBTQ terminology. |
2.94 (0.73) |
4.33 (0.69) |
1.39 |
5.9022 |
.001 |
|
S6 – I can recognize overt or obvious instances of oppression/discrimination |
4 (1.14) |
4.72 (0.46) |
0.72 |
2.4964 |
.0102 |
|
S7 – I can recognize subtle instances of oppression/discrimination (such as microaggressions) |
3 (1.14) |
3.79 (.73) |
0.78 |
2.4393 |
.011 |
As evident in the table, all seven statements came out with statistically significant positive change at the 95% significance level and five with a 99% significance level. This is the first time that we have run statistics like this on our survey data. To see this change is very affirming and let’s us know that what we are doing is working.