Información en Español
FAN2011402.JPG
FAN2011402.JPG
FAN2011402.JPG
FAN2011402.JPG
FAN2011402.JPG

Design Intro


Research is only as good as its design.

SCROLL DOWN

Design Intro


Research is only as good as its design.

The MSL foundation is characterized by high psychometric rigor and clear conceptual and theoretical grounding. Since its inception, the MSL has strived to generate data and results that are accurate, able to be applied with confidence, and add to the collective knowledge-base on leadership development. 

FAN2011402.JPG

Conceptual model


Conceptual Model

Conceptual model


Conceptual Model

 

An adapted version of Astin’s (1993) “input-environment-outcome” (I-E-O) college impact model provides the conceptual framework for MSL. This involves collecting data about students’ knowledge and experiences prior to college as well as their experiences during college. These can be triangulated to then examine the influences of experiences during college on a variety of educational outcomes.

The model is adapted in two key ways:

  1. The environment is extended to include variables representing experiences outside the college context (e.g., mentoring from employers, involvement in off-campus organizations).
  2. The study collects data at a single point with pre-college data collected through retrospective questions. Students are asked to think back to before they started college to capture these data points. This approach is supported by prior research indicating it reduces response-shift-bias and yields accurate indications of student gains (Howard, 1980; Rohs, 2002; Rohs & Langone, 1997).
FAN2011402.JPG

Theoretical Framework


Theoretical Framework

Theoretical Framework


Theoretical Framework

More than the Social Change Model

The original theoretical framework for the MSL was the Social Change Model of Leadership Development (SCM; HERI, 1996). It was selected given the SCM:

  • Is consistent with contemporary leadership theory across a wide-array of disciplines;
  • Was created specifically for use in working with college students; and
  • Is consistently named as one of the most well-known and applied student leadership models (Kezar, Carducci, & Contreras-McGavin, 2006; Owen, 2012)

The SCM measures socially responsible leadership capacity defined as “a purposeful, collaborative, values-based process that results in positive social change” (Komives, Wagner, & Associates, 2009, p. xii).  Its central principles are assessed through core values that describe students’ levels of self-awareness and abilities to work with others (HERI, 1996).

FAN2011402.JPG

Framework Evolution-Coparison


As THE MSL HAS EVOLVED, the theoretical framework has been adapted to capture the increasing complexity of the study. The current theoretical framework is still nested in the values of the SCM, but draws from a much wider set of theoretical bases

Framework Evolution-Coparison


As THE MSL HAS EVOLVED, the theoretical framework has been adapted to capture the increasing complexity of the study. The current theoretical framework is still nested in the values of the SCM, but draws from a much wider set of theoretical bases

This includes:

  • Contemporary Leadership Theory– Research from MSL has demonstrated that post-industrial theories of leadership have much in common. Thus, the theoretical framework now draws on key dimensions from a wider range of contemporary theories.
  • Social Psychology & Human Development– Research from MSL points to the complexity of developmental processes. Contributing to leadership development involves much more than simple skill-building. As such, the theoretical frame relies heavily on insights from psychological and human development theories to inform the study.
  • Critical & JusticeBased Perspectives– The MSL design has always attempted to model the values of social justice on which the SCM is predicated. This has led to increasing attention to design elements that draw on critical and justice-based perspectives.
 
 
MSL-theoretical-comparison.png
 
Model Constructs Does MSL Measure? Direct or Indirect How?
Authentic
Leadership
Self-Awareness Directly ✔ Consciousness of Self
✔ Collective Racial Esteem
✔ Motivation
✔ Self-Directed Learning
✔ Resiliency
Internalized Moral Perspective Directly ✔ Congruence
✔ Commitment
✔ Resiliency
✔ Cognitive Skills
Balanced Processing Directly ✔ Cognitive Skills
✔ Social Perspective Taking
Relational Transparency Indirectly ✔ Consciousness of Self
✔ Leadership Efficacy
✔ Resiliency
✔ Congruence
Relational
Leadership
Model
Purpose Directly ✔ Commitment
✔ Common Purpose
Process Directly ✔ Common Purpose
✔ Collaboration
✔ Controversy with Civility
Inclusive Indirectly ✔ Collaboration
✔ Social Perspective Taking
Ethical Indirectly ✔ Consciousness of Self
✔ Congruence
✔ Citizenship
Empowering Indirectly ✔ Collaboration
Emotionally
Intelligent
Leadership**
Consciousness of Context indirectly ✔ Belonging / Climate
✔ Cognitive Skills
✔ Social Perspective Taking
✔ Directed Learning
Consciousness of Self Directly ✔ Consciousness of Self
✔ Collective Racial Esteem
✔ Motivation
✔ Resiliency
✔ Congruence
Consciousness of Others Directly ✔ Common Purpose
✔ Collaboration
✔ Social Perspective Taking
✔ Controversy with Civility
Servant
Leadership
Conceptualizing Indirectly ✔ Common Purpose
✔ Cognitive Skills
Emotional Healing Indirectly ✔ Social Perspective Taking
Putting Followers First
Helping Followers Grow & Succeed
Behaving Ethically Indirectly ✔ Consciousness of Self
✔ Congruence
✔ Citizenship
Empowering Indirectly ✔ Collaboration
Creating Value for the Community Directly ✔ Citizenship
Leadership
Practices
Inventory**
Challenge the Process Indirectly ✔ Change
✔ Cognitive Skills
✔ Leadership Efficacy
✔ Controversy with Civility
Create a Shared Vision Directly ✔ Common Purpose
✔ Collaboration
Enable Other to Act Indirectly ✔ Collaboration
Model the Way Indirectly ✔ Congruence
✔ Commitment
✔ Resiliency
Encourage the Heart Indirectly ✔ Collaboration
Transformational
Leadership
Idealized Influence / Charisma Indirectly ✔ Congruence
✔ Commitment
Inspirational Motivation Indirectly ✔ Common Purpose
✔ Collaboration
Intellectual Stimulation Indirectly ✔ Controversy with Civility
✔ Common Purpose
✔ Cognitive Skills
Individualized Consideration Indirectly ✔ Social Perspective Taking
✔ Collaboration

** Leadership Practices Inventory and Emotionally Intelligent Leadership are copyrighted by John Wiley & Sons, Inc.

Instrument & Psychometrics


Instrument & Psychometrics

Instrument & Psychometrics


Instrument & Psychometrics

The MSL survey questionnaire is composed of both existing and newly created scales designed to measure elements of the conceptual and theoretical frameworks. The amount of data generated for use in evidence-based practice is significant and transferable across a variety of areas within higher education (e.g., leadership programs, residence life, multicultural affairs, academic units, institutional research, campus recreation, fraternity and sorority life).

The MSL survey is particularly useful because it:

  • Captures one of the widest ranges of student demographic data modeling the justice approach embedded in the theoretical framework. This includes capturing data on racial group membership, ethnic sub-group membership, AND racial identity salience; inclusion of identifiers for gender and sexual identity; a wide range of religious affiliations; and multiple dimensions of ability status.
  • Examines both collegiate experiences AND educational outcomes including a wide-range of identity and human development constructs (e.g., complex cognitive skills, resilience, social perspective-taking, spiritual development).

The MSL survey instrument generates over 400 items and scales. The information below provides samples of key variables from across three domains of the study:

 

Input Variables

Capture student demographics as well as knowledge and experiences prior to enrollment in college. Variables include:

  • Age
  • Gender and sexual identity
  • Racial and ethnic group membership
  • Military status
  • Parental education and income
  • Pretest measures for all educational outcomes
  • Involvement experiences prior to higher education

Experiences During College

Capture a wide range of experiences that occur during college and are often aligned with high-impact educational practices. Variables include:

  • Mentoring relationships (e.g., with faculty, peers, community members, employers, parents)
  • Academic-based experiences (e.g., study abroad, first-year seminars, research with faculty, internships)
  • Involvement experiences (e.g., breadth and depth of involvement in both on and off-campus organizations)
  • Civic engagement involvement (e.g., type and levels of involvement in community service learning experiences)
  • Leadership development experiences (e.g., levels of involvement in both curricular and co-curricular leadership programs)
  • Interactions about and across difference and perceptions of campus climate

Outcomes

Capture the degree of achievement across a wide-array of educational and leadership-related outcomes. This includes measures of:

  • Leadership capacity (i.e., the knowledge, skills, and abilities associated with socially responsible leadership) including self-awareness, values congruence, ability to collaborate effectively, community engagement, and navigating conflict with civility among others;
  • Leadership efficacy (i.e., one’s internal belief in their ability to lead effectively);
  • Leadership behaviors (i.e., the degree to which on actually enacts leadership capacity to advance social responsibility and change);
  • Higher order cognitive abilities including complex cognitive skills and social perspective-taking;
  • Developmental outcomes related to resilienceracial identity and spiritual development; and
  • Sense of belonging on campus.
 

Note that MSL also makes use of “sub-studies” as part of the research design. This involves administering some variables to a randomly selected 50% of the participants at each campus. MSL uses sub-studies to explore new areas of interest without dramatically increasing the length of the survey.

The MSL survey includes more than 400 variables, scales, and composite measures.  As such, it would be impossible to detail full information related to the validity and reliability of measures.  Much of this information can be found in academic articles published using the various scales accessible via this website.  Additionally, a brief summary of the psychometrics of the survey can be found here.

The MSL survey instrument is also constantly evolving to include important new variables associated with student inputs, experiences during college, and outcomes. The survey instrument is typically finalized in June of the year prior to administration. If you would like to see a sample of the MSL survey instrument please contact MSLConnection@gmail.com.

Sampling


Sampling

Sampling


Sampling

So, what does it look like for a campus to participate in the MSL?

The MSL is administered online by SoundRocket, LLC. (formerly Survey Sciences Group, LLC).  SoundRocket is an independent research organization with specific expertise in multi-campus studies.  All data are collected using empirically proven standards for web-based survey research.

MSL’s online format allows students to share their experiences at any time or place that is convenient to them. 

They can respond to the MSL when they feel comfortable, secure, and unhurried. We’ve partnered with SoundRocket to also conduct “studies of the study.” This allows us to better target data collection based on empirical evidence of how best to captures students’ information with minimized burden and maximized rates of return.

The data-collection period extends from January though the end of April, and each institution selects a three-week window that best fits their unique academic calendar.  Students are invited to participate via personalized e-mails.  Each correspondence outlines the study, addresses issues of confidentiality and consent, and supplies a link to the survey instrument.  Students may receive up to four total contacts.

As the survey is administered, SoundRocket provides comprehensive respondent support via e-mail or telephone to minimize the inconvenience to your students.  The service also allows students to opt out of the survey by replying to any MSL correspondence and requesting to be removed from the dataset.  SoundRocket will monitor the survey completion rates and work with you to encourage participation if the response is insufficient.

To increase your response rate, you may host a sweepstakes-style drawing for students who complete the survey.  Your institution will decide on the number, type, and value of the prizes; and SoundRocket will conduct the random drawing.  Additionally, the MSL offers a number of monetary prizes raffled at the national level to stimulate survey response.

Data Security


Data Security

Data Security


Data Security

SoundRocket fully commits to confidentiality and security of survey data. We approach security in two ways:

  • Protocols for maintaining confidentiality of survey participants and their data, and
  • Technical systems that prevent unwanted access of data from outsiders

Confidentiality

Protecting the confidentiality of study participants is the most important concern of the SoundRocket staff. All individuals employed at SoundRocket are bound by confidentiality as condition of their employment. They have signed a pledge of confidentiality and have been trained in procedures for maintaining confidentiality and privacy.

Data are stored on SoundRocket internal servers that are password protected. The survey data captured in the web-based interface do not have any personal identifiers (individual or company name, email address, other contact information) in the data files. Personal identifiers are kept in a separate sample file in a secure space on our local file server accessible only by staff who require this information as part of their work. We never rent, sell, or give your personal information to any third party for the purpose of directly marketing any products or services to you. Any information you provide in the survey is used strictly for research purposes.

Data Security

We establish both logistical and physical barriers to protect respondent data to ensure its secure transmission and storage. Data are received and transmitted via password-protected, 128-bit SSL technology. Survey data submitted are encrypted before transmission via the SoundRocket web site from participants’ PCs/laptops to a secured server at SoundRocket . Data are archived in secure servers, accessible through password-protected networks by appropriate personnel. We use both stand-alone networks and firewalls to safeguard data against outside networks attacks.

The security of the information on our system has never been compromised.

Many websites use “cookies” to store information about a user in order to expedite the completion of forms on future visits to that site. SoundRocket does not use cookies on our website. The only information we collect from you on our website or any web-based survey is the information you provide.

SoundRocket monitors industry alerts and trends to ensure that we abide by the latest laws and regulations in protecting respondent data. We rigorously uphold — and often surpass — industry standards for research ethics, privacy, and confidentiality.

If you have questions about data confidentiality and security, please contact us:

E-mail: privacy@soundrocket.com

Phone: (734) 527-2180  and request to speak with our Privacy, Confidentiality and Ethics Officer.