FINAL ASI cover

Toolkit for Evaluating Positive

Youth Development

THE AFTER-SCHOOL INITIATIVE’S

The Colorado Trust is a grantmaking foundation dedicated to advancing the health and well-being of the people of Colorado. To learn

more about The Colorado Trust and its grantmaking initiatives, go to www.coloradotrust.org.

“The Colorado Trust” is registered as a trademark in the U.S. Patent and Trademark Office. After-School Initiative’s Toolkit for Evaluating

The Colorado Trust is pleased to have organizations or individuals share its materials with others. To request permission to excerpt from

this publication, in print or electronically, please contact Dr. Nancy B. Csuti, Senior Evaluation Officer, The Colorado Trust, 1600

Sherman Street, Denver, CO 80203-1604; fax 303-839-9034; e-mail [email protected].

Please cite this summary as: The Colorado Trust. After-School Initiative’s Toolkit for Evaluating Positive Youth Development. Denver, CO: The

Colorado Trust; 2004.

The

After-School Initiative’sToolkit for Evaluating Positive Youth Development

was developed by The Colorado Trust and National

Research Center, Inc.

The Colorado Trust

1600 Sherman Street

Denver, CO 80203-1604

303-837-1200

Fax: 303-839-9034

www.coloradotrust.org

National Research Center, Inc.

Measuring what matters

3005 30th Street

Boulder, CO 80301

303-444-7863

Fax: 303-444-1145

www.n-r-c.com

e-mail: [email protected]

Table of Contents

INTRODUCTION .............................................................................................................................................................................1

TOOLKIT BACKGROUND................................................................................................................................................................................1

PHILOSOPHY BEHIND THE TOOLKIT ..............................................................................................................................................................2

BENEFITS OF USING THE TOOLKIT ................................................................................................................................................................4

USING THIS TOOLKIT................................................................................................................................................................... 5

RECRUITMENT, INCENTIVES AND CONSENT ..................................................................................................................................................5

UNDERSTANDING THE RIGHTS OF THE SURVEY PARTICIPANTS.....................................................................................................................6

DEVELOPING THE SURVEY............................................................................................................................................................................6

DEVELOPING THE SURVEY ADMINISTRATION PROTOCOL .............................................................................................................................8

PILOT TESTING THE SURVEY.......................................................................................................................................................................12

THE SURVEY ADMINISTRATION ..................................................................................................................................................................13

AFTER THE SURVEY ADMINISTRATION.......................................................................................................................................................15

SECTION 1: POSITIVE YOUTH DEVELOPMENT DOMAINS................................................................................................16

ACADEMIC SUCCESS ...................................................................................................................................................................................19

ARTS AND RECREATION..............................................................................................................................................................................21

COMMUNITY INVOLVEMENT.......................................................................................................................................................................22

CULTURAL COMPETENCY ...........................................................................................................................................................................23

LIFE SKILLS ................................................................................................................................................................................................25

POSITIVE CORE VALUES .............................................................................................................................................................................26

POSITIVE LIFE CHOICES ..............................................................................................................................................................................27

SENSE OF SELF............................................................................................................................................................................................28

SECTION 2: PROGRAM QUALITY...............................................................................................................................................29

SECTION 3: YOUTH BACKGROUND AND PROGRAM PARTICIPATION...........................................................................33

SECTION 4: ALTERNATIVE TO POST-ONLY SURVEY ..........................................................................................................35

Toolkit for Evaluating Positive Youth Development

APPENDIXES ..................................................................................................................................................................................36

APPENDIX I. TOOLKIT DEVELOPMENT ........................................................................................................................................................36

APPENDIX II. SELECTING A SAMPLE OF PARTICIPANTS TO SURVEY ............................................................................................................40

APPENDIX III. THE USE OF INCENTIVES IN EVALUATION............................................................................................................................41

APPENDIX IV. CONSENT AND ASSENT FORMS ............................................................................................................................................42

APPENDIX V. ENGLISH AND SPANISH COMPARISON OF QUESTION SETS.....................................................................................................50

APPENDIX VI. PROTOCOL WORKSHEET FOR ASI TOOLKIT SURVEY ADMINISTRATION ..............................................................................56

APPENDIX VII. TOOLKIT OUTCOME DOMAINS LINKED TO POSITIVE YOUTH DEVELOPMENT MODELS ......................................................58

APPENDIX VIII. ACADEMIC SUCCESS TRACKING FORMS............................................................................................................................74

APPENDIX IX. ..................................................................................................79COMPLETE POST-ONLY TOOLKIT SURVEY QUESTION SET

APPENDIX X. DETERMINING WHICH SURVEY TO USE: POST-ONLY VERSUS PRE-POST ...............................................................................92

APPENDIX XI. COMPLETE PRE-POST TOOLKIT SURVEY QUESTION SET......................................................................................................99

Introduction

The After-School Initiative’s Toolkit for Evaluating Positive Youth Development contains a number of evaluation question sets that staff of an after-

school program may find useful to assess youth outcomes. Because after-school programs’ goals and activities are often unique, the toolkit

is not intended to provide all the evaluation questions program staff would need to demonstrate success. Instead, it provides question sets

to measure outcomes common to many after-school programs aimed at promoting positive youth development. Outcomes are the

measurable changes in participants’ knowledge, attitude or behavior (e.g., 50% of youth participants reported they spent more time doing

their homework because of the after-school program).

The questions in this document are designed to measure 45 different youth outcomes within 8 outcome sets or “domains” (Section #1):

1. Academic success

2. Arts and recreation

3. Community involvement

4. Cultural competency

5. Life skills

6. Positive life choices

7. Positive core values

8. Sense of self.

In addition, a question set to measure the quality of after-school programs is included (Section 2), as well as a question set on youth

background and program participation.

Toolkit Background

The After-School Initiative’s Toolkit for Evaluating Positive Youth Development is a resource of The Colorado Trust’s After-School Initiative (ASI).

This 5-year (2000-2005), $11 million After-School Initiative serves children in fourth through ninth grades through programs that promote

positive youth development. National Research Center, Inc. (NRC) developed the toolkit with funding from The Colorado Trust as part of

the After-School Initiative’s evaluation technical assistance to grantees. The outcomes selected for this toolkit were based on a synthesis of

research on surveys used to measure assets and positive youth development in the United States; a review of the positive youth

development literature; the logic models from 35 ASI grantees; and surveys and focus groups with youth-serving programs. Surveys

developed from this toolkit were pilot tested in 4 after-school programs serving youth ages 8 through 15 and representing African-

American, Caucasian/white, Latino and Native American youth. (See Appendix I for more details on the toolkit development.)

Toolkit for Evaluating Positive Youth Development 1

Toolkit for Evaluating Positive Youth Development 2

Philosophy Behind the Toolkit

The original goal of this project was to provide ASI grantees with surveys that were research-based or often used by other large youth-

serving organizations. Through research, NRC found that many of the evaluation surveys used to measure outcomes in youth were not

age-appropriate, culturally sensitive or realistic in terms of the burden on respondents or staff resources necessary to use them in an on-

going evaluation process. After reviewing many of the existing question sets for assessing youth outcomes, NRC decided to develop its

own sets of questions and adapt questions from other surveys to better meet ASI grantees’ needs. The following criteria were used to

develop this toolkit:

Gathering the opinions of youth: Using this toolkit allows adherence to principles of positive youth development — “youth as

resources” and “adult-youth partnerships” — by asking youth to report how they feel about programs and how their lives have been

changed as a result of participation in specific programs.

Age and literacy appropriateness: The survey questions provided in this toolkit were developed to be age appropriate for youth in

fourth through ninth grades so that the opinions of all youth served in ASI programs could be measured. The questions were written for

youth as young as fourth grade and were simply stated so program participants with lower literacy levels can complete them with little help.

However, a number of the outcomes that were found to be important in positive youth development programming are difficult to translate

into questions at a fourth-grade level (e.g., cultural competency). Questions associated with these outcomes are noted by a ¿ in each

section of the toolkit or separated into basic and advanced sets of questions so program staff can choose appropriate set for the group of

young people served by individual programs. As a general rule, NRC found that each of the questions with a ¿ were better understood by

youth in grades 6 and above and less comprehensible to youth below the sixth grade.

Cultural appropriateness: Although it is impossible to develop a set of questions that is not influenced by some form of cultural bias,

NRC attempted to develop questions appropriate for most youth in fourth through twelfth grades. The question sets have been pilot tested

with youth of various ages, ethnicities and cultures to help further decrease cultural bias. However, program staff is encouraged to look

closely at whether or not the question sets fit their program and the cultural background of participants and modify question sets

accordingly.

Mitigation of burden on participants and staff: For an evaluation to be sustained within a program, the evaluation must be manageable

for both the program staff collecting the data and the program participants providing the data. The survey question sets are short so that

after-school programs can measure multiple outcomes without undue burden on youth participants. The survey is easy to administer and

involves minimal training of program staff. Finally, the survey questions are straightforward so data analysis and reporting are not complex.

Measuring relevant results: Because every ASI program has unique goals and activities, not all will achieve the same outcomes. NRC

encourages program staff to use logic models to determine the outcomes that best fit with the services they provide. Measuring outcomes

in an area where little or no services are provided not only wastes valuable staff time but may also provide less than desirable results.

Sensitivity to change: Many ASI programs attempt to foster significant, positive changes in the lives of their youth participants. These

changes often occur incrementally over time, so it is important to be able to measure small changes. However, small changes are often

difficult to measure. Many of the methods commonly used to measure change violate evaluation principles for youth, (e.g., using 4 or more

scale options and using higher literacy adjectives to differentiate scale points), making it more difficult to demonstrate small increments of

change. For this reason, “post-only” questions are provided in this toolkit. “Post-only” implies that program staff administers surveys at

the end of the program or after a “significant” amount of intervention has occurred (i.e., school-based program staff may want to

administer in December and May every year). “Post-only” surveys do not include survey administration before programs begin (as there

would be in a pre-post survey). “Post-only” surveys ask program participants to report whether they have changed at the end of a program,

instead of measuring outcomes before and after an intervention and comparing the results to detect changes.

Augmenting the use of existing question sets: There are scores of surveys designed to measure many of the same outcomes proposed

for measurement in this toolkit. This toolkit was not developed to replace the use of such surveys because many of the surveys used in

academia and clinical settings have been found to be valid and reliable ways of measuring youth characteristics. If this toolkit does not

include enough questions to provide the details desired to measure a specific program focus, staff may want to consider augmenting the

question sets and creating a longer, more specific survey.

Toolkit for Evaluating Positive Youth Development 3

Toolkit for Evaluating Positive Youth Development 4

Benefits of Using the Toolkit

This toolkit offers ASI program staff the evaluation materials necessary to measure many of the outcomes they plan to achieve. The toolkit

contains question sets and tracking forms that are straightforward in measuring a multitude of youth outcomes in a simple fashion. Other

benefits of using the toolkit include:

Specific question sets for the After-School Initiative programs: This toolkit was developed specifically for ASI

programs, taking into account the types of services provided and the youth served. Many question sets in the existing

literature were developed for other purposes and for other populations.

Timesaving: It is easier to use existing question sets than to reinvent the wheel. Often, after-school programs do not

institutionalize evaluation systems because staff gets bogged down creating and/or selecting the best measurement question

set. By selecting outcomes of interest to a program, staff can create a customized survey with relatively little work.

Pooled data: ASI programs will have a way to compare their results to other programs in the initiative. After analyzing the

data, NRC will send each grantee a customized report that compares the responses of youth served by the grantee’s

program to responses of all youth served by the After-School Initiative.

Using This Toolkit

As part of their evaluation technical assistance to ASI grantees, NRC provides an evaluation liaison for each ASI program. The evaluation

liaison serves as a consultant to guide and assist program staff to develop a program logic model, identify the program’s youth development

outcomes, develop a survey and survey administration protocol, and understand and use the results of the survey.

With their evaluation liaison, staff at each after-school program should review the outcome domains and question sets provided in this

toolkit and determine whether they are applicable to their program. In addition, staff should consider whether the question sets reflect the

youth development outcomes that individual programs are working toward according to their logic model. Once outcome domains and

question sets are chosen, NRC, through the evaluation liaisons’ communication with ASI programs, will customize surveys for programs

electronically. Liaisons will help ASI programs determine the most appropriate dates for survey administration and develop protocol for

administration. Then, after program staff administers the surveys to their youth participants, completed surveys will be sent to NRC for

analysis. Finally, within a few months of administration, evaluation liaisons will meet with their assigned ASI programs to review the data

results and discuss ways of using their data for program improvement.

Recruitment, Incentives and Consent

Determine who should be recruited to complete the survey: Every youth in fourth through ninth grades participating in after-school

programs funded by ASI is eligible to be a survey respondent. Whenever possible, all eligible participants should be asked to complete the

survey. If resources permit, it is easier to survey all participants to avoid designing special data collection methods that survey only a

representative sample of participants. If resources do not permit, the evaluation liaison should help program staff determine a sampling

method. Appendix II provides instructions for developing a sampling scheme for those programs unable to survey all participants.

Decide if incentives will be offered: Program staff needs to decide if incentives will be offered as part of the survey administration

process. Appendix III provides more information on the use of incentives to increase participation in evaluation.

Obtain consent and assent for the evaluation: The consent and assent forms are designed to protect evaluation participants by

informing them of their rights as well as any risks and benefits that may occur as a result of participation in the evaluation. Participants

must understand what the evaluation is about and what they are being asked to do. Program staff is required to have a signed current

consent and assent form on file for each participant in the evaluation.

Before an ASI program collects data from youth – even if it is anonymous – program staff must have a consent form signed by a parent or

guardian of each youth on file. The consent form is intended to describe the risks and benefits of participation in the evaluation and to give

Toolkit for Evaluating Positive Youth Development 5

Toolkit for Evaluating Positive Youth Development 6

parents and youth the option NOT to participate. In addition, child assent forms should be signed by all youth participating in the

evaluation. Consent and assent forms approved by The Colorado Trust and required for all ASI programs are provided in Appendix IV.

NRC evaluation liaisons may assist ASI program staff in customizing the forms for use with their individual programs.

Understanding the Rights of the Survey Participants

Respect for the participant is the cornerstone of ethical research, including maintaining the confidentiality of participants. If a participant

discloses his/her survey responses, program staff is not allowed to discuss that information with anyone unrelated to the evaluation. Even

if a respondent asks program staff to disclose an answer given in the survey, staff is not permitted to do so.

Confidentiality: Program staff should remind participants of their confidentiality rights whenever it is appropriate, and refer to

participants’ consent and assent forms, which guarantees these rights. The only exception to confidentiality: If the respondent shares

information revealing that they might harm themselves or others, then program staff must follow procedures that are legally mandated to

report such information.

Voluntary Participation: Evaluation participants cannot be forced to complete a survey. Instead program staff should stress the

importance to the program in learning how best to deliver after-school services. Participants are free to not answer individual questions and

their involvement in or withdrawal from the evaluation has no impact on the services they receive.

Developing the Survey

The questions included in this toolkit are quantitative in nature with fixed-response options (except for a few open-ended questions).

Individual ASI programs may want to augment these quantitative data with other forms of more qualitative data and anecdotal information

such as journals, photos or narratives depicting what youth participants have learned. Some helpful guidelines are:

Keep the survey short and to the point: The simpler the survey, the more likely it will be understood and completed. To enhance

simplicity, only question sets that pertain to a particular program should be used. In addition, program staff should only use question sets

that are at literacy and comprehension levels appropriate for the youth those programs serve.

Guarantee anonymity or confidentiality: Programs may choose to have surveys administered anonymously or confidentially. Anonymity

means that absolutely no identifying information will be collected from respondents. Confidentiality indicates that programs enforce clear

rules prohibiting unauthorized staff access to any information that would identify a particular respondent. NRC recommends administering

surveys anonymously using no personal identifiers when using toolkit questions in a “post-only” format. ASI program staff should refer to

consent forms when determining whether to make the survey anonymous or merely confidential.

Begin the survey with more general, less threatening questions: Place the most sensitive or personal questions closer to the end of

the survey. This includes questions on socio-demographic characteristics such as age, race or ethnicity, and questions within the positive

life choices outcome domain.

Make the survey friendly and attractive: Surveys that are well laid out and logical will ensure higher response rates. Use caution when

adding unnecessary mood boosters like cartoon graphics that might bias results. Here are some suggestions to make the survey more

appealing to youth:

 Use appropriate size fonts (i.e., 11 point or 12 point font).

 Use interesting, but easy-to-read fonts (i.e., Comic Sans MS, Kristen ITC or Pooh, rather than Jokerman).

 Make sure the order of questions is logical and easy to follow. For younger audiences, consider adding the leading stem of the

question to each question instead of only listing it at the start of a question set.

 Avoid designing a survey that looks like a “test.”

 Print surveys on pleasingly colored paper.

 Avoid overcrowding of questions — allow enough “white space” on each page.

 If the number of questions makes the survey too long for one implementation, create more than one survey and administer it on

different days.

In addition, ASI program staff may want to customize questions from this toolkit so individual participants can relate to them. For

instance, many program quality questions use the word “staff.” Programs may want to change this to program leader, teacher or some

other term used at the program to mean those individuals working with the youth. Other ideas for customization are provided throughout

the toolkit as “helpful hints.” The wording of questions, however, should not be changed if changes would impact the nature of the

question. NRC evaluation liaisons will help program staff make these determinations as surveys are drafted. ASI program staff is advised to

consult with their NRC evaluation liaison to customize toolkit surveys electronically.

Toolkit for Evaluating Positive Youth Development 7

Toolkit for Evaluating Positive Youth Development 8

Determine if a Spanish version of the survey is needed: All toolkit question sets have been translated into Spanish and back translated

into English to ensure the accuracy of the translation. See Appendix V for a comparison of the English and Spanish language questions.

ASI program staff is advised to consult with their NRC evaluation liaison to customize Spanish language toolkit surveys electronically.

Developing the Survey Administration Protocol

To ensure that a program receives consistent data from an evaluation survey, reliable data collection procedures must be in place.

Consistent data collection is important to guarantee accurate, trustworthy results. For each survey or other data collection question set

being used, the following questions must be answered. ASI programs should consult their NRC evaluation liaison for help answering these

questions and completing the protocol:

• Who is eligible?

All youth participants in fourth through ninth grades for whom a program has signed evaluation consent and assent forms on file are

eligible to complete the survey.

• Will all eligible youth participants be surveyed?

Programs serving large numbers of youth (more than 100) may want to consider developing a sampling plan to survey fewer than the

total number of participants. ASI program staff considering this option should consult the NRC evaluation liaison and review

Appendix II. If a program collects data from only one group of participants, program staff should make sure these young people are

similar to the youth who will not be surveyed.

• When will data be collected?

For most programs using this toolkit, data will be collected at program completion before year’s end. Program staff should decide the

exact date when surveys will be distributed and filled out by participants. If programs collect data over a short time period, programs

should make sure the young people attending are similar to the young people that may be missed.

• Who is responsible for gathering supplies (e.g., survey copies, pencils, survey collection box) and making sure the

surveys are complete?

Assign duties and name names.

• What explanatory text is provided along with the survey question set?

Will all staff administering the survey use similar instructions? If the intention is to collect responses anonymously from youth, make

sure they know that. How are completed surveys returned? Make sure youths’ anonymity is not threatened. Have staff agree not to

hover around youth as they complete the survey question set. If possible, have the survey returned to a neutral staff member who will

not be affected by youths’ answers or allow youth to deposit the survey in a protected box.

• How will the survey be administered?

Most programs will administer their survey by having participants fill them out on their own with paper and pencil or pen. Programs

with youth who have limited English or low-literacy skills may choose to read questions and response options aloud to their youth.

Youth can then follow along and complete the survey using their individual paper survey copies.

• How much time will be allotted for participants to complete the survey?

Provide an adequate amount of time for completion so that youth do not feel rushed or that they are missing out on other program

activities.

• Will a survey be provided and administered in Spanish as well as in English?

If so, a separate protocol may be needed for administering the Spanish version.

• How will candid responses be ensured?

To increase the likelihood of honest responses, NRC recommends that surveys designed from this toolkit be administered

anonymously. That means no one will know the responses provided by each youth participant. To make sure a participant’s anonymity

is not threatened, consider the following methods:

 Have a volunteer who is neutral hand out surveys

 Use a ballot-style box for collection of surveys

 Have staff agree not to hover around participants while they complete the surveys

 If a ballot box is not feasible, have the survey returned to a staff member who will not be affected by the participants’ answers.

Toolkit for Evaluating Positive Youth Development 9

Toolkit for Evaluating Positive Youth Development 10

When developing a protocol, the ages and literacy levels of participants should be taken into account. Program staff may want to consider

the following methods for younger and older youth.

Method

Younger (4

and 5

grade literacy)

Older (6

grade and above literacy)

Administration Method

Group administration (questions read

aloud while participants answer with

pencil and paper in group setting)

Self-administered (hand out in group setting or

upon entrance and or exit of program)

Question Topics

Select from questions without a ¿ -

indicating a more basic literacy level

Can choose from all questions

Use of Open-ended Questions

Only with a pilot test

Yes, if desired

Survey Length

Keep it to 1-2 pages

Keep it to 4 pages

The following is a checklist of tasks to consider before the survey administration:

Question Response

1. Do you know which youth participants are eligible to take the

survey?

 Yes  No

2. When will data be collected?

Date: ________

Time: From ________ to ________

3. Who will administer the survey?

Name: ____________________

4. Have you trained the survey administrator(s) on what

explanatory text to recite when distributing surveys to youth?

 Yes  No

5. How much time is allotted to survey administration? (Provide

an adequate amount of time for completion so that youth do not feel

rushed or that they are missing out on other program activities.)

6. Who is responsible for gathering supplies?

Name: ____________________

7. Do you have pens/pencils ready for administration?

 Yes  No

8. Do you have enough surveys copied for the administration in

both English and Spanish (if needed)?

 Yes  No

A “Protocol Worksheet for ASI Toolkit Survey Administration” is provided in Appendix VI. This worksheet should be completed in

consultation with the evaluation liaison.

Toolkit for Evaluating Positive Youth Development 11

Toolkit for Evaluating Positive Youth Development 12

Pilot Testing the Survey

Once program staff has developed the survey and a survey administration protocol, the survey can be pilot tested with a small group of

youth participants prior to surveying the larger group. Even though NRC pilot tested toolkit questions prior to releasing this toolkit, staff

of individual programs should test each customized survey to make sure it will work for the youth at their specific programs.

To conduct a pilot test, program staff should select 5 to 10 of their program participants who vary in terms of age, literacy, gender,

race/ethnicity and any other characteristics that may influence the way someone may respond to or perceive the evaluation question sets.

Program staff should pilot test the survey question set with each respondent individually, asking him or her not only to complete the survey

or interview but to also “think aloud” while completing it (note: this may be difficult for a younger respondent). The respondent might tell

staff, “I’m not sure what this question is asking but I think it’s asking this…” or “I don’t know this word.” The information staff receives

from pilot test respondents should help them decide whether the survey question set is appropriate for a particular age group or culture;

whether it answers the questions intended; and whether it is easy for respondents to understand. In addition, staff should time how long it

takes each respondent to complete the survey. After completing the survey, staff should ask the respondent a number of questions:

1. Overall, how did you feel about the questions in the survey?

2. Were there any parts that were hard or confusing?

3. Were there any parts of the survey that made you feel uncomfortable or any questions that you felt like you didn’t want to answer?

4. Did you or do you think other kids would feel angry being asked any of the questions in the survey?

5. Do you think other kids would be tempted to answer less than truthfully on any of these questions? If so, which ones?

6. How do you think other kids in the program might feel about taking this survey?

Program staff may also want to ask additional questions specific to topics or items that youth participants seemed uncertain about. After

discussing the survey, staff should look over the completed surveys to make sure respondents answered the questions in an appropriate

manner. Staff may find that some questions need additional work.

The Survey Administration

The survey administrator is a key player in documenting the program’s success. The survey administrator is responsible for effectively

collecting information on the program. This includes making sure that participants understood the purpose of the survey, were comfortable

completing surveys and that all surveys were filled out properly.

NRC recommends that programs choose a survey administrator who is not directly responsible for the after-school program, because the

youth participants may not feel comfortable indicating their feelings about the activities, staff or their actions. If there is no other person

besides program staff who can reasonably administer the surveys, then it is imperative that the survey administrator encourages participants

to give accurate and straightforward answers to the survey and to stress the privacy of the survey process.

Survey administrator guidelines

Train the survey administrator:

Prior to administering the survey with youth participants, the survey administrator should be

trained on the toolkit. In addition, the survey administrator should be familiar with the specific survey question set. Survey

administrators should complete the survey themselves, reading the directions out loud. To anticipate participant questions regarding

the survey, survey administrators can role-play questions and answers and then discuss and agree upon consistent responses to

common questions in advance. For example, youth may want to know the difference between “kind of” and “not really.”

Standardize the administration process:

To ensure that information collected from all participants is comparable, the survey

administration should be standardized — surveys should be administered to each participant in the same way as much as possible.

Survey participants should be given adequate time and privacy to respond to the survey. For self-administered surveys, someone

should be available to answer any questions or clear up any confusion participants might have about the survey. For interviewer-

administered surveys (an option for lower literacy youth), the interviewer should read the questions as written, in the order they are

presented in the survey question set, and should allow participants to pick their answers without influence by the interviewer. Also,

in both situations, but particularly in the interviewer-administered situation, the survey administrator should not react, either

positively or negatively, to any attitudes or behaviors that a respondent reveals.

Maintain neutrality toward participants:

To ensure that participants do not feel compelled to answer questions in a particular

way to please the interviewer or administrator, the survey administrator should keep a neutral tone and expression while still being

friendly and professional. Survey administrators should not be judgmental regarding participants’ knowledge, attitudes or behavior;

any judgments survey administrators do have should not be detectable by participants.

Toolkit for Evaluating Positive Youth Development 13

Toolkit for Evaluating Positive Youth Development 14

Survey administration procedures

1. Prepare all materials prior to the scheduled survey.

2. Leave enough time to set up and arrange the room appropriately prior to the participants’ arrival.

3. Introduce yourself to the participants as they arrive and give a brief description of your purpose for being there and administering

the survey. Use the sample script below as a guide to introduce yourself and the evaluation to participants, but make sure the

language is your own. Also make sure you convey all the major ideas reflected in the script, but use your own style of

communication to ensure that you are able to establish rapport with the participants from the beginning.

4. Hand out surveys to participants.

5. Request that they attempt to answer every question in order that their opinions and experiences can be fully included in the

evaluation.

6. Let participants know you are here to help them if they have any questions. They should raise a hand to indicate they need you.

7. While participants are completing the survey, quietly walk around the room (do not hover), answer questions and make sure things

are running smoothly.

8. When youth finish the survey, thank them for their time and provide incentives, if appropriate.

SAMPLE SCRIPT

Hi, I’m (insert your name here). I am here to ask you to participate in our after-school program evaluation.

The survey I am about to pass out is being used to get your ideas about our after-school program. It is not a

test and there are no right or wrong answers — we want to know what your experiences and opinions are.

Take your time and be sure to answer each question based on what you really think. If you cannot read or

don’t understand a question, please raise your hand or come find me and I can help you.

The survey should take you only 8 to 10 minutes. (May differ depending on survey length.)

Make sure you do NOT put your name on the survey so that we can keep your answers private.

Okay, let’s begin.

Problem situations during survey administration

1. If a participant appears to be going too slowly or too quickly, discretely check on them. If they need additional assistance, you may

want to offer to read them the questions in another room.

2. If a participant does not understand a question or a response:

a. Read the question or response aloud to them

b. Emphasize key words that may help convey meaning

c. Ask if there are any words that the participant does not understand

d. Try to paraphrase the word or phrase. Give word or phrase options until the participant understands the question; however,

keep as close to the original question as possible without leading participants to any particular response.

e. If a participant is concerned about who will see answers remind them that only members of the program team and NRC

evaluators will see their individual surveys. No one will ever see his or her name attached to the survey and no information will

be presented about individual respondents; only group averages will be used in reports.

After the Survey Administration

Once ASI program staff develops and administers the surveys, staff should submit data to NRC for analysis. After data are analyzed,

evaluation liaisons will meet with program staff to review data and discuss ways of using results for program improvement. Specific

instructions for submitting completed surveys to NRC are:

1. Make copies of the surveys before sending them to NRC in case they get lost in the mail.

2. Include the survey administration tracking form (to be completed with an NRC evaluation liaison).

3. Mail the packet of completed surveys to:

Laurie Urban

National Research Center, Inc.

3005 30th Street

Boulder, CO 80301

4. Send an e-mail to your NRC liaison saying that a batch of surveys has been mailed so that NRC can be on the lookout for them.

5. Once data analysis is complete, evaluation liaisons will schedule a time with ASI program staff to review the data.

Toolkit for Evaluating Positive Youth Development 15

Toolkit for Evaluating Positive Youth Development 16

Section 1: Positive Youth Development Domains

This toolkit contains 8 sets of outcomes or “domains”: academic success, arts and recreation, community involvement, cultural

competency, life skills, positive life choices, positive core values and sense of self (see graphic below). The specific outcomes included in

each domain are presented in the table on the following page. (For further reference, Appendix VII links the outcome domains and

questions to positive youth development models.)

Sense of Self

Toolkit

Outcomes

Positive

Core Values

Positive

Life Choices

Life Skills

Cultural

Competency

Community

Involvement

Arts and

Recreation

Academic

Success

Outcome Domain Outcome Indicators

Academic success

• Grade improvement

• Educational skill improvement: math, science, language arts, CSAP scores

• School attachment

• School engagement, involvement

• Interest in learning (e.g., reading for pleasure)

• Graduation rates

Arts and recreation

• Appreciation for arts (and crafts), music, dance or theater

• Performance in arts (and crafts), music, dance or theater

• Performance in sports, exercise and recreation activities

• Time spent in arts, music, dance or theater

• Time spent in sports, exercise and recreation activities

Community involvement

• Time spent in community service

• Sense of importance to community (youth as resources)

• Self-concept due to community involvement

Cultural competency

• Respect for others

• Comfort with others

• Knowledge of other cultures, races or ethnic groups

• Respect for other cultures, races or ethnic groups

• Comfort with other cultures, races or ethnic groups

• Knowledge of own culture

• Interest in own culture

• Sense of belonging to cultural group

• Pride in one’s culture

Toolkit for Evaluating Positive Youth Development 17

Toolkit for Evaluating Positive Youth Development 18

Outcome Domain Outcome Indicators

Life skills

• Friendship skills

• Goal-setting skills

• Problem-solving skills

• Conflict resolution skills

• Teamwork

• Communication skills

• Decisionmaking skills

• Planning for the future

• Leadership skills

Positive life choices

• Resistance skills

• Restraint: drug abuse, tobacco use, alcohol use, violent behaviors, early sexual

activity

Positive core values

• Caring

• Empathy

• Integrity

• Honesty

• Responsibility

• Equality and fairness

Sense of self

• Self-concept (self-confidence, self-esteem, self-worth)

• Empowerment

• Positive outlook

• Sense of purpose

Academic Success

Many after-school programs focus on improving academic success. Academic success involves both improved academic outcomes (e.g.,

grades, Colorado Student Assessment Program (CSAP) scores, graduation rates, school attendance), as well as improved attitudes about

school (e.g., school engagement, class participation, time spent on homework, motivation for learning). Programs that have strong

partnerships with schools may be able to obtain records, such as grades and CSAP scores, directly from schools. These programs should

use the tracking forms provided in Appendix VIII. Although the use of school-provided data may be the best measure of academic

performance, programs without access to school data can use this question set to obtain participants’ self-report of changes in academic

performance, as well as other academic-related attitudes and behaviors.

ASI requirement: If the academic success domain is selected, all questions in bold (a, e, i and j) are required. Note: Questions in bold on

page 20 also are required.

Academic Success

Coming to [this program] has helped me to…

YES KIND OF NOT REALLY

a. Improve my grades in school ....................................................................... ...................... .............................

b. Improve my grade in language arts (reading/writing)................................... ...................... .............................

c. Improve my grade in math................................................................................ ...................... .............................

d. Improve my grade in science ............................................................................ ...................... .............................

e. Do better in school .......................................................................................... ...................... .............................

f. Do better in language arts (reading/writing).................................................. ...................... .............................

g. Do better in math ............................................................................................... ...................... .............................

h. Do better in science............................................................................................ ...................... .............................

i. Try harder in school........................................................................................ ...................... .............................

j. Spend more time doing my homework ..................................................... ...................... .............................

Toolkit for Evaluating Positive Youth Development 19

Toolkit for Evaluating Positive Youth Development 20

Report of academic-related behaviors

There is more to academic success than grades, test scores and attendance. Many after-school programs aim to improve attitudes about

school and behaviors related to learning. This question set contains questions on the topics of school attachment (e.g., bonding to school,

classmates and teachers), school engagement (e.g., trying “harder,” spending more time on homework, attending class more often) and the

promotion of a general interest in learning (e.g., reading for pleasure).

ASI requirement: If the academic success domain is selected, all questions in bold (a-f) are required. Note: Questions in bold on page 19

also are required.

Attitudes and Behaviors Related to Academic Success

Coming to [this program] has helped me to…

YES KIND OF NOT REALLY

a. Participate more in class activities ............................................................. ...................... .............................

b. Become more interested in going to school ............................................ ...................... .............................

c. Care more about my school .......................................................................... ...................... .............................

d. Get along better with my classmates ......................................................... ...................... .............................

e. Get along better with my teachers .............................................................. ...................... .............................

f. Spend more time reading for fun (not for school).................................. ...................... .............................

Arts and Recreation

Many youth programs focus on building skills through activities in arts, music, drama, dance or recreation. This question set contains

questions that measure both time spent in the arts and recreation as well as increased performance. Appreciation for the arts is also covered

in the question set.

Helpful hint: Limit arts and recreation questions to those that pertain specifically to the program. For example, do not include questions

about “dance” if the program does not purposely involve youth in dance.

ASI requirement: If the arts and recreation domain is selected, a program may choose to use questions a through c, d through f and/or

g through i.

Arts and Recreation

Because I came to [this program]…

YES KIND OF NOT REALLY

a. I spend more time doing arts and crafts ......................................................... ...................... .............................

b. I enjoy arts and crafts more ............................................................................. ...................... .............................

c. I am better in arts and crafts ............................................................................. ...................... .............................

d. I spend more time in music, dance and/or theater ....................................... ...................... .............................

e. I enjoy music, dance and/or theater more...................................................... ...................... .............................

f. I am better at music, dance and/or theater .................................................... ...................... .............................

g. I spend more time doing sports, exercise and/or

recreation activities ............................................................................................. ...................... .............................

h. I enjoy sports, exercise and/or recreation activities more............................ ...................... .............................

i. I am better at sports and/or recreation activities........................................... ...................... .............................

Toolkit for Evaluating Positive Youth Development 21

Toolkit for Evaluating Positive Youth Development 22

Community Involvement

Providing opportunities for youth to contribute to their communities is a focus of many after-school programs. This question set contains

questions about time spent in volunteer activities, feeling more connected to the community, feeling more like a community resource

(youth as resources) and improved self-concept due to time spent in service to others.

ASI requirement: If the community involvement domain is selected, all questions are required.

Community Involvement

Coming to [this program] has helped me to…

YES KIND OF NOT REALLY

¿ a. Spend more time volunteering or helping

others in my community .................................................................... ...................... .............................

¿ b. Spend more time looking for opportunities

to help others in my community...................................................... ...................... .............................

¿ c. Feel more important to my community......................................... ...................... .............................

¿ d. Feel a stronger connection to my community............................. ...................... .............................

¿ e. Feel better about myself because I help others........................... ...................... .............................

¿ Indicates this question is appropriate only for older or more advanced youth.

Cultural Competency

As communities are becoming more diverse, knowledge, comfort and respect for others who are different than oneself becomes

increasingly important. There are many definitions of cultural competency; the definition used in The Colorado Trust’s After-School

Initiative is:

Cultural competency is an ongoing process and practice that builds the capacity of organizations and individuals to understand, accept, value and honor the unique

contributions of all people, including but not limited to people’s: ability, age, disability, ethnicity, gender, gender identity, geographic region, health, language, mental

health, race, religion, sexual orientation, socioeconomic status and spirituality.

This question set contains questions on general feelings of respect for and comfort with others who are thought to be “different,” as well

as questions that focus more specifically on culture, race and ethnicity as a cultural group. Questions within the culture, race and ethnicity

category assess comfort and respect for other cultures as well as knowledge and pride in one’s own culture. Note that all of the questions

related specifically to culture, race and ethnicity as a form of cultural competency are listed within the advanced question set, indicating that

they are more appropriate for older youth. Many younger participants are not yet aware of cultural, racial or ethnic terminology.

ASI requirement: If the cultural competency domain is selected, programs may choose either the basic or advanced question sets.

Questions in bold are required.

Cultural Competency (Basic Set)

Because I came to [this program]…

YES KIND OF NOT REALLY

a. I know more about the good things that people

who look or sound different than me have done

(like African Americans, Mexican Americans,

Native Americans, Asian Americans and others)................................... ...................... .............................

b. I talk more with young people who look or sound

different than me ............................................................................................. ...................... .............................

c. I feel more comfortable being around young

people who look or sound different than me ........................................... ...................... .............................

d. I understand that someone who looks or sounds

different than me may not be that different after all.............................. ...................... .............................

Toolkit for Evaluating Positive Youth Development 23

Toolkit for Evaluating Positive Youth Development 24

Cultural Competency (Advanced Set)

Being involved in [this program] has helped me to…

YES KIND OF NOT REALLY

a. Know more about people of other cultures, races

or ethnic groups ............................................................................................... ...................... .............................

b. Care more about young people of other cultures,

races or ethnic groups .................................................................................... ...................... .............................

c. Have more respect for young people of other

cultures, races or ethnic groups than me.................................................. ...................... .............................

d. Feel more comfortable with young people

of other cultures, races or ethnic groups................................................... ...................... .............................

e. Talk more to young people who speak languages

other than English at home .......................................................................... ...................... .............................

f. Try harder not to judge people based on skin color.............................. ................. .......................

g. Be more aware that I sometimes have ideas about

people that are not true (incorrect or mistaken)..................................... ...................... .............................

Because I came to [this program]…

YES KIND OF NOT REALLY

a. I know more about my own culture, race or ethnic group .................. ...................... .............................

b. I am more interested in my own culture, race or ethnic

group.................................................................................................................... ...................... .............................

c. I feel more connected to my family’s cultural traditions...................... ...................... .............................

d. I feel more connected to my own culture, race or ethnic

group.................................................................................................................... ...................... .............................

e. I feel more pride for my own culture, race or ethnic group................. ...................... .............................

f. I have more respect for members of my own community ................... ...................... .............................

Life Skills

Life skills are proficiencies that allow youth to transition into and achieve successful adulthood. They are often important to educational

attainment, employment and interpersonal relationships. This question set contains questions on decisionmaking, planning, leadership,

conflict resolution, friendship skills, teamwork and communication.

Helpful hint: Many of the life skills questions may have a “ceiling” effect with youth participants. In other words, a participant may

already feel like he or she is very good at a particular skill prior to and without influence by an after-school program. If this is the case,

these survey questions may not be sensitive enough to show significant increases over time. If many of your young people come to a

program with high levels of these skills and the interventions are not intense, program staff may not see significant change due to program

activities.

ASI requirement: If the life skills domain is selected, all questions are required.

Life Skills

Because I came to [this program]…

YES KIND OF NOT REALLY

a. I get along better with other people my age............................... ...................... .............................

b. I am better at making friends ......................................................... ...................... .............................

¿ c. I am better at telling others about my ideas and feelings ...... ...................... .............................

d. I am better at listening to other people........................................ ...................... .............................

e. I work better with others on a team.............................................. ...................... .............................

¿ f. I make better decisions .................................................................... ...................... .............................

¿ g. I am better at planning ahead ........................................................ ...................... .............................

¿ h. I am better at setting goals.............................................................. ...................... .............................

¿ i. I am better at solving problems ..................................................... ...................... .............................

j. I am more of a leader ........................................................................ ...................... .............................

¿ k. I am better at taking care of problems without

violence or fighting............................................................................ ...................... .............................

¿ Indicates this question is appropriate for older or more advanced youth.

Toolkit for Evaluating Positive Youth Development 25

Toolkit for Evaluating Positive Youth Development 26

Positive Core Values

Some of the most important traits after-school programs nurture in youth relate to the instillment of positive values such as caring,

empathy, integrity, honesty, responsibility, equality and fairness. These are the values most often taught in many of the large national youth-

serving organizations such as the YMCA, Girl Scouts, Boy Scouts, 4-H and Boys and Girls Clubs.

Helpful hint: Although positive core values in the field of youth development are considered to be important, the inclusion of this

outcome domain in this toolkit is unusual because many of these values are often referred to as “traits” or “character” and are considered

by many to be predominantly developed through the first years of life. Moreover, youth and adults alike tend to believe (or at least report)

that they care about people, tell the truth and stand up for what they believe. Very few will self-report to not upholding these values. This is

not to say that these values are unchangeable or that after-school programs do not influence these values, but programs without significant

interventions in these areas may find little success at changing them in a measurable way. Thus, NRC encourages programs to use these

questions but consider wisely the population served and decide if the program offers “enough” intervention to warrant a noticeable change.

ASI requirement: If the positive core values domain is selected, questions a, b, c and d are required for basic-level programs.

All questions are required if a program considers its participants more advanced.

Positive Core Values

Because I came to [this program]…

YES KIND OF NOT REALLY

a. I care more about other people ...................................................... ...................... .............................

b. I care more about the feelings of other people.......................... ...................... .............................

c. I am better at standing up for what I believe............................. ...................... .............................

d. I tell the truth more often even when it is hard ......................... ...................... .............................

¿ e. I am better at taking responsibility for my actions .................. ...................... .............................

¿ f. I am more interested in community and world

problems ............................................................................................... ...................... .............................

¿ g. I am better at speaking up for people who have

been treated unfairly.......................................................................... ...................... .............................

¿ Indicates this question is appropriate for older or more advanced youth.

Positive Life Choices

Although the focus of positive youth development is to emphasize assets and strengths of youth rather than deficits, it is important to

understand if programs have helped to change undesirable behavior in youth. Likewise, some after-school programs are funded in order to

move youth away from substance use, violence and sex during the high-risk after-school hours. The positive life choices questions included

in this question set are general in nature, focusing on resistance skills and restraint rather than the number of times each undesirable

behavior occurs.

ASI requirement: If the positive life choices domain is selected, questions in bold (a, b and g) are required.

Positive Life Choices

Being involved in [this program] has helped me to…

YES KIND OF NOT REALLY

¿ a) Better say “no” to things I know are wrong............................... ...................... .............................

¿ b) Stay out of trouble.............................................................................. ...................... .............................

¿ c) Stay away from tobacco ...................................................................... ...................... .............................

¿ d) Stay away from alcohol ........................................................................ ...................... .............................

¿ e) Stay away from drugs............................................................................ ...................... .............................

¿ f) Stay away from sex................................................................................ ...................... .............................

¿ g) Stay away from violence and fighting .......................................... ...................... .............................

Note: An additional scale option of “not interested” may be added.

¿ Indicates this question is appropriate for older or more advanced youth.

Toolkit for Evaluating Positive Youth Development 27

Toolkit for Evaluating Positive Youth Development 28

Sense of Self

The sense of self outcome domain relates to how youth view themselves and their abilities to cope with the basic challenges of life. There

are many ways to think about the self and many different words to describe it: self-esteem, self-concept, self-worth, self-evaluation, self-

assurance, self-regard, self-efficacy and the like. For the purposes of this toolkit, 4 concepts of self are covered: self-concept, sense of

empowerment, positive outlook and sense of purpose.

Self-concept might be defined as a conscious, cognitive perception of how someone sees him or herself.

It is often used interchangeably

with self-esteem, but tends to define a more global

and potentially more changeable notion

of self. NRC chose to use the word self-

concept rather than self-esteem for a number of reasons:

1. The term “self-esteem” has been used so frequently in mainstream culture that it has become trivialized and misinterpreted.

2. There is much debate over the definition, the causes and the importance of self-esteem.

3. In NRC’s understanding of the constructs, self concept is more transient, thus more susceptible, to changes from the environment.

Increased empowerment is the feeling of competence or ability to cope with life’s challenges. Terms such as locus of control, self-efficacy,

resiliency and personal power represent alternative words that describe allied traits. Positive outlook relates to a person’s optimism about

his or her personal future and sense of purpose is the feeling that one’s life has meaning.

ASI Requirement: If the sense of self domain is selected, all questions are required.

Sense of Self

Coming to [this program] has helped me to…

YES KIND OF NOT REALLY

a. Feel better about myself................................................................................. ...................... .............................

b. Feel I have more control over things that happen to me ..................... ...................... .............................

c. Feel that I can make more of a difference................................................ ...................... .............................

d. Learn I can do things I didn’t think I could do before......................... ...................... .............................

e. Feel better about my future........................................................................... ...................... .............................

f. Feel I am better at handling whatever comes my way.......................... ...................... .............................

Section 2: Program Quality

Toolkit questions on program quality were developed to assess youth’s reactions to staff and activities, and to collect program ratings often

associated with best practices in after-school services and general youth programming. The topics included are based on a review of the

literature on positive youth development and successful youth programs

4, , , , ,5 6 7 8 9

and include many of the external assets from Search

Institute’s assets framework

. Questions about external assets are directly related to the program and staff rather than parents, family,

school and community, as these are domains over which many programs have little control. The questions included in this portion of the

toolkit are core elements of quality, so they apply to all after-school programs, regardless of the activities provided. All programs are

encouraged to include program quality questions in all surveys administered to youth. Program quality topics are listed in the table below

and the questions used in this question set follow.

Program Quality Topics

Supportive, Caring Climate

(Positive Adult Relationships)

Youth as Partners and

Resources

Safety and Trusting

Environment

Attractive and Meaningful

Activities

• Young person receives support from

non-parent adults

• Staff can be trusted, care about and

respect youth

• Staff is well liked by youth

• Youth feel welcome at program

• Adults recognize when young person

does good things

• Staff encourages young person to do

well

• Youth are given useful roles in

program

• Youth voices are listened to

when planning content and

processes

• Young people have significant

roles in making decisions for

program

• Youth have significant roles in

doing the activities they decide

to do

• Young people are empowered

to enhance their role as

community resources

• Young person feels safe

• Program has clear rules and

consequences

• Staff sets rules and establishes

clear norms of behavior

• Staff recognizes, values and

responds to diverse

backgrounds and experiences

of youth

• Young person is willing to seek

advice and counsel from staff

• Tailored to interest of youth

• Programs teach “life skills” (measured

in the Life Skills section of the toolkit)

• Youth learn new skills

Toolkit for Evaluating Positive Youth Development 29

Toolkit for Evaluating Positive Youth Development 30

Helpful hint #1: Questions marked advanced may be appropriate for younger youth if the questions are read aloud to participants and

explanation is given to clarify harder-to-understand words or concepts.

Helpful hint #2: Carefully select the questions that apply to your program. Do not include questions that do not apply. For example,

program staff may not regularly spend time with youth one-on-one or encourage youth to do volunteer work. If that is the case with your

program, do not include these questions in your survey.

ASI requirement: All program quality (Set A) questions are required.

Program Quality (Set A)

Please check the box below that is closest to how you feel about [this program].

YES KIND OF NOT REALLY

a. There are interesting activities at [this program]................................... ...................... .............................

b. The activities are fun at [this program]..................................................... ...................... .............................

c. I learn about new things at [this program] .............................................. ...................... .............................

d. I like coming to [this program] ................................................................... ...................... .............................

e. There are rules I am expected to follow at [this program].................. ...................... .............................

f. I feel safe at [this program]........................................................................... ...................... .............................

g. I feel like people are happy to see me here .............................................. ...................... .............................

h. I tell my friends to come to [this program].............................................. ...................... .............................

ASI requirement: All program quality (Set B) questions are required, except for questions d and l.

Program Quality (Set B)

Please check the box below that is closest to how you feel about the staff at [this program].

YES KIND OF NOT REALLY

a. Staff cares about me........................................................................................ ...................... .............................

b. Staff helps me feel important........................................................................ ...................... .............................

c. Staff gives me lots of support ....................................................................... ...................... .............................

d. Staff spends time with me one-on-one ........................................................... ...................... .............................

e. Staff can be trusted.......................................................................................... ...................... .............................

f. I like the staff here........................................................................................... ...................... .............................

g. Staff expects me to try hard to do my best ............................................... ...................... .............................

h. Staff tells me when I do a good job ............................................................ ...................... .............................

i. I could go to a staff member at [this program] for

advice if I had a serious problem ................................................................ ...................... .............................

j. Staff listens to what I have to say ................................................................ ...................... .............................

k. Staff asks me to plan, choose or lead activities....................................... ...................... .............................

l. Staff encourages me to do volunteer work..................................................... ...................... .............................

m. Staff treats all kids fairly................................................................................. ...................... .............................

Toolkit for Evaluating Positive Youth Development 31

Toolkit for Evaluating Positive Youth Development 32

Open-ended Questions

The foundation for this toolkit is closed-ended or fixed-choice questions because research on youth finds that open-ended questions often

challenge the cognitive abilities of adolescents and younger teens. Further, fixed-choice questions provide uniformity in response that

makes them easier to analyze, aggregate and interpret. However, the information provided in open-ended questions provides richer detail

on why youth rate programs as they do, which can be very important for program improvement. Thus, this toolkit includes a couple of

generic open-ended questions on program quality that programs may want to include if they feel their youth can provide meaningful

responses and will not have difficulty completing the questions.

Open-ended Questions on Program Quality

What do you like best about coming to [this program]?

__________________________________________________________________________________________

If you could change one thing about [this program] what would it be?

__________________________________________________________________________________________

Section 3: Youth Background and Program Participation

Program staff should include questions in the survey pertaining to youth participants’ background. However, programs should not include

this question set if they are concerned about surveying only a small number of youth; in small numbers, youth respondents may feel they

can be identified by age, gender or race/ethnicity (e.g., if only two girls are surveyed and one of the girls identifies herself as Latina).

ASI requirement: All background information questions are required for advanced-level programs. Basic-level programs may choose not

to ask the question on race/ethnicity. In addition, programs may choose to ask when a participant started coming to the program in

seasons or in months. Younger audiences may not yet understand the concept of months.

Background Information

How often do you come to

[this program]?

 Every day or almost every day

(4 to 5 times a week)

 2 to 3 times a week

 Once a week

 A couple times a month

 Less than once a month

How old are you?

 6 years or younger

 7 years

 8 years

 9 years

 10 years

 11 years

 12 years

 13 years

 14 years

 15 years or older

When did you start coming

to [this program]?

 This fall (this spring)

 Over the summer

 Last spring (last fall)

 More than 1 year ago

 Within the last 3 months

 4 to 6 months ago

 7 to 12 months ago

 Last year or longer ago than that

What grade are you

in? ______________

Are you a girl/female or a

boy/male?

 Girl/female

 Boy/male

Which best describes you?

 Caucasian/White

 African American/Black

 Asian/Pacific Islander

 Latino/Hispanic

 Native American

 Bi-racial/Multi-racial

 Other ______________

Toolkit for Evaluating Positive Youth Development 33

Toolkit for Evaluating Positive Youth Development 34

Activity Identification

In the following question, youth participants are asked to check all the after-school program activities with which they are involved. (For

example, activities may include homework help, reading group, craft time, open gym, etc.) This allows program staff to use the same survey

with multiple activity groups and enables them to separate results for a variety of activities.

Activity Identification

Check each of the after-school activities you participate in at [this program].

 ________________________________________________________

Note: Customization is needed for each program.

Section 4: Alternative to Post-only Survey

Most ASI programs use the post-only survey. (See Appendix IX for the complete post-only survey question set. Note: On the Spanish

language version of the survey, section instructions are in English to allow non-Spanish speaking staff to customize the survey

appropriately for program participants.) However, due to the requirements of other funders, a few ASI programs requested use of the pre-

post survey. ASI programs administering the pre-post toolkit surveys are additionally required to administer a post-only survey at program-

end to permit calibration of the two methods. This calibration permits conversion of pre-post results to the post-only format so that all

results can be integrated and provided to ASI programs as normative comparisons. Appendix X, which includes pre-post survey questions

that correspond with the post-only question sets, is provided as a guide for determining whether the post-only survey method or pre-post

survey method is most appropriate for individual programs.

Toolkit for Evaluating Positive Youth Development 35

Toolkit for Evaluating Positive Youth Development 36

Appendixes

Appendix I. Toolkit Development

This toolkit was developed through the synthesis of a number of data collection processes including research on survey question sets used

to measure assets and positive youth development in the United States, a review of positive youth development literature, integration of the

logic models from 35 ASI grantees, and the conduct of surveys and focus groups with Mile High United Way (MHUW) youth-serving

programs. MHUW programs received funding from The Colorado Trust to develop evaluations focused on positive youth development.

Since the goals of the MHUW grant and ASI were similar, they were included in the ASI toolkit development.

Each study component is detailed below.

National search for surveys and database of youth outcome questions

The first step of the toolkit development was to research survey question sets currently being used to measure positive youth development

outcomes. The search proved to be difficult because many of the question sets being used by youth-serving agencies are not published or

available on the Internet. (Since NRC’s original research was conducted, a new web site by RMC,

http://cart.rmcdenver.com, has appeared

which provides a broad listing of youth-oriented program outcome survey question set citations.) To accomplish the task, NRC staff:

• Reviewed all of the survey question sets developed by Search Institute

• Attempted to contact more than 300 agencies by Internet and telephone that are listed as partners with Search Institute or have

worked with Search Institute recently (based on the Search Institute website)

• Held teleconference with 10 United Way agencies across the nation and asked these agencies to forward any survey question sets or

leads

• Reviewed all of the survey question sets currently used by MHUW youth-serving agencies

• Contacted large national youth-serving programs to determine if they measure youth outcomes (Girl Scouts, Boy Scouts, Campfire

Boys and Girls, Boys and Girls Clubs, Big Brothers Big Sisters and 4-H).

In all, NRC made more than 300 contacts either through e-mail or telephone (approximately 100 e-mails were returned due to bad

addresses). NRC staff received more than 50 responses and located and reviewed more than 220 survey question sets as part of this search.

Over 2,500 individual questions were entered into a Microsoft Access database and linked to the ASI toolkit outcome domains.

Review of positive youth development literature

NRC staff reviewed current models of positive youth development as well as commonly used youth services models to assess the shared

themes in philosophy. The following models were reviewed as part of the toolkit development process: Search Institute, 8 Habits of the

Heart, Henderson and Milstein’s Resiliency Model, Individual Risk Factors and Protective Index, Bernard’s Resilience Assessment,

Targeting Life Skills Model, America’s Promise, SCANS, Cornerstones, Communities of Caring, Communities That Care, Carerra Model,

Character Counts and Choice Theory. NRC also looked at models developed by other large youth-serving organizations: 4-H 4 Fold

Model, Girls Scouts, YMCA and YWCA. Outcomes common in positive youth development were extracted for the toolkit.

Synthesis of After-School Initiative logic models

As part of NRC’s work with The Colorado Trust’s After-School Initiative, all 35 grantees completed logic models detailing the services and

expected outcomes. These outcomes were synthesized across grantees to determine which outcomes after-school programs shared. All of

the outcomes that were shared by 6 or more grantees were included in the toolkit. These outcomes were: improved academic performance

(e.g., improved GPA, improved CSAP scores, increased school attendance, increased graduation rates, increased time spent on homework),

increased community involvement (volunteerism), increased cultural competency, improved interpersonal skills (e.g., social skills,

relationships with peers, etc.), increased self-esteem, and positive life choices (e.g., change in attitude, knowledge or behavior in regard to

substance use, sex, violence, etc.).

Survey of youth-serving agencies

NRC surveyed 25 MHUW youth-serving agencies in August and September 2001 to determine the services they were providing, the

philosophies and frameworks by which they were guided and outcomes they hoped to achieve. NRC also assessed the evaluation work

currently being done by 30 youth-serving agencies and the evaluation supports that they needed. The survey results helped NRC staff

define the outcomes of interest and the positive youth development models to be included in the toolkit. It also helped NRC to better

understand the need of many youth-serving programs to have a question set that would assess youth outcomes.

Focus group with youth-serving agencies

NRC conducted a focus group in March 2002 with MHUW youth-serving agency staff to get their ideas on a draft evaluation question set

using many of the questions contained in this toolkit with a focus on measuring Search Institute assets. The grantees in attendance included

representatives from YouthBiz, Mi Casa, Asian Pacific Development Center, Big Brothers Big Sisters of Colorado, YMCA, Colorado I

Have a Dream, and Boys and Girls Clubs of Metro Denver. In addition, NRC staff called additional grantees to gauge their interest in the

draft question sets and methods (Annunciation School; Denver Area Council, Boy Scouts of America; Northwest Coalition 4 Better

Schools; and the Washington Street Community Center).

Toolkit for Evaluating Positive Youth Development 37

Toolkit for Evaluating Positive Youth Development 38

The results from the focus groups and telephone interviews helped NRC refine ideas on the use of a “toolkit” rather than a single, uniform

survey that cannot be modified because of the various goals and outcomes unique to each program, the importance of keeping the question

sets age and culturally appropriate and the need to make these question sets simple to administer and complete.

Blending the studies

After completing all 5 of these tasks, NRC compiled all of the information and selected the outcomes and methods that appeared to be of

most use to a large variety of youth-serving organizations.

Pilot testing the question sets

The items in this toolkit have been pilot tested with the help of 4 after-school programs: Cross Community Coalition’s Super Scholars

Program, Cross Community Coalition’s Leadership Group, Southwest Mercy Housing’s Holly Park after-school program and BASE

Bulldog After-School Enrichment Program. These programs serve youth ages 8 through 15 who are African-American, Latino, Native

American and White/Caucasian. The results of the pilot test were used to alter questions and question sets to be better understood by the

youth respondents served.

Reliability analysis of the toolkit

Reliability analysis of the outcome domains within the toolkit was conducted to determine the internal consistency of the questions or

items within each domain. Internal consistency is an indicator of how well the different questions measure the same concept. In general, a

scale should have an alpha coefficient of .7 or greater before it is widely used

, although Robinson et. al.

, use the following rating criteria

for evaluating alpha-coefficients: exemplary .80 or better; extensive .70 to .79; moderate .60 to .69; and minimal <.60. Using the criteria

from Robinson et. al., all of the scales or domains in the toolkit have extensive to exemplary internal consistency.

The reliability analysis was conducted using customized toolkit surveys administered by ASI programs by June 2003 (Total N=257; n’s for

each sub-domain ranged from about 43 to 159) with considerable variability among responding youth in age, ethnicity and gender.

Reliability of Toolkit Domains

Domain

Scale/Domain

Number of

Items

Alpha

Coefficient

Program quality

Attractive and meaningful activities

6 .78

Supportive positive adult relationships

9 .91

Safe and trusting environment

6 .84

Academic success

Grade improvement

4 .87

Skill improvement

4 .74

School engagement, involvement

8 .88

Arts and recreation

Arts and crafts

3 .81

Music, dance and theatre

3 .87

Sports and recreation

3 .81

Community involvement 5 .88

Cultural competency

Respect, comfort, knowledge of other cultures (advanced)

5 89

Respect, comfort, and knowledge of other cultures (basic)

3 .83

Knowledge, interest, sense of belonging, pride in one’s culture

5 .92

Life skills

Friendship and communication

5 .86

Decisionmaking, planning and leadership

6 .85

Positive life choices

Restraint from delinquent behavior

7 .92

Positive core values 6 .88

Sense of self 6 .88

Toolkit for Evaluating Positive Youth Development 39

Toolkit for Evaluating Positive Youth Development 40

Appendix II. Selecting a Sample of Participants to Survey

If resources do not permit all participants to be surveyed, a sampling scheme can be developed. (“Sample” is a term that as a noun refers to

a subgroup of a total group, or as a verb refers to selecting a subgroup.) If sampling is used, be sure that the sample selected is

“representative” of all youth participants, that is, that the participant profile (such as age, race/ethnicity of the participant, etc.) of the

sample is similar to the profile of all participants.

“Systematic” sampling is often the easiest way to choose a sample. To select a systematic sample, divide the total number of participants by

the number to be surveyed. The nearest whole number will be the “skip factor.” For example:

If a program has the resources to survey 50 youth participants,

And there are 230 participants at the program,

The skip factor would be 230 ÷ 50 = 4.6

If 4 is used as the skip factor, there will be about 57 or 58 participants to survey.

(230 ÷ 4 = 57.5)

If 5 is used as the skip factor, there will be about 46 participants to survey.

(230 ÷ 5 = 46)

Once the skip factor is chosen (a program could use either 4 or 5 in the example above), the list of participants can be used to select the

sample. A random number between one (1) and the skip factor should be chosen as the place to begin on the list. For example, if the

number three (3) is chosen as the first number, start with the third participant on the list. That participant would be the first to include in

the survey. Then count down the list by the skip factor, selecting every “Nth” (where N equals the skip factor) participant. These are the

participants to survey.

Appendix III. The Use of Incentives in Evaluation

An ASI program may choose to use incentives as a way of enticing youth to participate in their evaluation or as a way of getting parents to

return consent forms. Incentives are a good way to increase participation and demonstrate that a program values youth participants’ and

parents’ time. One of the downsides of incentives is cost. Also, some researchers argue that incentives may introduce more bias because

they compel participants to answer more favorably due to payment or other reward received or may select for specific sub-populations

based on the type of incentive.

Many programs may not need incentives because youth participants and parents are often motivated and place a high value on the

program. In this case it may be unnecessary to provide incentives for participation in evaluation activities.

However, some programs may need incentives to help increase participation in their efforts. If low response rates are expected or become

an issue, incentives can be used to increase participation. Incentives can be used in a variety of ways:

 Advance payment for future work: incentives are given to all participants (or parents) asked to take part in the evaluation regardless

of their response. These incentives are often less significant and play more on the “guilt factor” rather than reward people with

reasonable compensation for the work performed. (This approach is often used by charitable organization asking for donations.

For instance, personalized address labels, pens, a penny, etc. are mailed to motivate people to give.)

 Payment for work completed: participants who complete the survey are given an incentive. These incentives are often more

significant such as t-shirts, hats, snacks, bumper stickers, coupons for use at a fast-food restaurant, etc. An option for programs

with limited budgets is the use of a lottery or drawing. Every participant completing a survey is entered in the lottery to win a prize.

Note that if incentives are paid for work completed, there must be a system in place to protect the anonymity or confidentiality of the

respondents (i.e., people cannot be asked to sign the survey so they get their prize). A common way to get around this dilemma is to ask

individuals to turn in a second form indicating the survey is complete at the same time the actual completed survey is returned. The surveys

and completion forms are kept in two different boxes.

Instead of offering money as an incentive, consider alternatives. Some ASI programs have given youth candy (or more healthy treats) as a

reward for completing the survey. Others have enticed youth with an end-of-semester pizza party. In any case, program staff needs to make

sure youth understand that their reward is for participation in the survey and NOT for giving high marks.

Toolkit for Evaluating Positive Youth Development 41

Toolkit for Evaluating Positive Youth Development 42

Appendix IV. Consent and Assent Forms

Following are the consent and assent forms approved by The Colorado Trust and required for all ASI programs. NRC evaluation liaisons

may assist ASI programs in customizing the forms for use in their individual programs. Spanish language consent and assent forms are

provided after the English language consents.

English Language Consent and Assent Forms

[Name of ASI program/agency] PROGRAM EVALUATION CONSENT

The after-school program that your child participates in is funded in part by The Colorado Trust. [Name of

ASI program/agency], in partnership with The Colorado Trust and National Research Center, Inc. (NRC)

will evaluate [name of ASI program] periodically throughout the year. We are hopeful that the information

gained in this evaluation will help us improve programming for your child by demonstrating the

effectiveness of [name of ASI program] and help us learn if participants are gaining positive youth

development skills due to their participation in the program. Because the program evaluation will collect

survey responses and develop knowledge, the program evaluation involves research. As part of our

evaluation, your child will be asked to complete a short survey. All evaluation materials collected will be

released to The Colorado Trust and National Research Center, Inc. for their evaluation use and assistance

with analysis.

Questions concerning your child’s opinions about the program as well as personal feelings and activities will

be included in the survey. Questions will focus on your child’s satisfaction with the program, how safe

he/she feels while at the program and how program staff responds to him/her. Depending on the particular

focus of program activities for which your child is involved, he/she will additionally be asked questions

pertaining to positive youth development.

Examples of questions are listed below.

• I feel safe at this program (Yes, Kind of, Not Really)

• Program staff listens to what I have to say (Yes, Kind of, Not Really)

• This program helped me to feel better about myself (Yes, Kind of, Not Really)

• This program helped me to spend more time reading (Yes, Kind of, Not Really)

[ASI program staff may delete some or all of the questions below if they will not be asking questions related

to such topics]

• Being involved in this program has helped me to stay away from tobacco (Yes, Kind of, Not Really)

• Being involved in this program has helped me to stay away from drugs (Yes, Kind of, Not Really)

• Being involved in this program has helped me to stay away from alcohol (Yes, Kind of, Not Really)

• Being involved in this program has helped me to stay away from sex (Yes, Kind of, Not Really)

Your child’s participation in this evaluation is strictly voluntary. Your child is free to refuse participation.

Your child can skip (not answer) questions that he/she does not wish to answer. Participation in the

evaluation is not required nor does it affect program attendance in a positive or negative way. In addition, if

your child leaves or is absent (including voluntary withdrawal, temporary absence, suspension, expulsion or

termination) from [name of ASI program] he/she may still be asked to participate in the evaluation.

There may be some risk to your child from participating in this evaluation. Some of the questions we ask

may be embarrassing or make youth participants feel uncomfortable. For example, a child may not want to

respond to a question about their self-esteem or personal behaviors. As stated above, your child has the

right to skip (not answer) any of the questions that make him/her embarrassed or uncomfortable. This

program evaluation has been designed specifically for children and is not unlike questions that your child

might encounter in school or other such programs. However, if you are at all concerned, you are

encouraged to contact the after-school program staff to review the evaluation materials.

All information your child provides for evaluation will be used for statistical and evaluative purposes in such

a manner that [Choose either: (your child cannot be identified. Your child’s name will not appear anywhere

in reports generated by [name of ASI program/agency], NRC or The Colorado Trust. All information your

child provides for evaluation will be kept confidential.) (OR) (All information your child provides for

evaluation will be confidential.)] There are two exceptions to the promise of confidentiality: If information is

revealed concerning suicide, homicide or child abuse and neglect, law requires that such information be

reported to the proper authorities. In addition, should any information contained in this evaluation be the

subject of a court order or lawful subpoena, [name of ASI program/agency] might be compelled to disclose

information in compliance with the subpoena.

[If applicable] In addition, the evaluation team of [name of ASI program/agency] may wish to examine

program participants’ school records (report cards, CSAP scores, scores on standardized tests, attendance

Toolkit for Evaluating Positive Youth Development 43

Toolkit for Evaluating Positive Youth Development 44

records). Your signature gives NRC and [name of ASI program/agency] staff the permission to access these

records.

If you have any questions regarding the evaluation or your child’s rights as an evaluation participant, or if

you wish to obtain a copy of the entire survey to be used with your child before signing this consent, please

contact [ASI program staff name] at [phone number] ___________________________________.

I have read and understand the above evaluation description. For the things I do not understand I have

asked for and received a satisfactory explanation. I agree to have my child participate in this evaluation

through May 2005, the end of The Colorado Trust’s After-School Initiative, and I understand that I may

withdraw my consent at any time without loss or termination of my child’s participation in the [name of ASI

program/agency]. I also understand that my child’s privacy will be protected [Choose either: (because my

child’s responses cannot be traced to my child.) OR (because my child’s responses will not be reported

individually.)]

Child Participant’s Name: (please print) ___________________________________________________

Parent’s/Guardian’s Name(s): (please print) ________________________________________________

Parent’s/Guardian’s Name(s) Signature: ___________________________________________________

(If child has two legal guardians, both signatures are highly recommended.)

Parent’s/Guardian’s Name(s): (please print) ________________________________________________

Parent’s/Guardian’s Name(s) Signature: ___________________________________________________

Date: ______________________________________________________________________________

Child Assent to Participate in the [name of ASI program/agency] Program

Evaluation

As a participant in [name of ASI program/agency], I agree to be in a study to answer some questions about

my activities, feelings and opinions about [name of ASI program] and how my participation has helped or

not helped me. I understand that my answers are private [Choose either: (and that my name will not go on

my answer sheet.) (OR) (and that my answers will only be reported as part of a bigger group and will not

reflect my individual answers.)] I will not get a grade for my answers nor will anyone discuss my answers

with me unless I ask them to. I agree to participate in this evaluation through May 2005 as long as I am

participating in [name of ASI program/agency]. I can decide to stop being in this evaluation at any time

without getting into trouble. If I do not answer the questions I will continue to be able to attend [name of

ASI program].

Child’s Name: (please print) _________________________________________________________

Child’s Signature: _________________________________________________________________

Date: ___________________________________________________________________________

Date of Birth: ____________________________________________________________________

Adult Witness*:______________________________________ Date: __________________________

Adult Witness Signature: ______________________________________________________________

*Witness must be present for the assent process.

[Program copy OR Child Participant copy]

Toolkit for Evaluating Positive Youth Development 45

Toolkit for Evaluating Positive Youth Development 46

Spanish Language Consent and Assent Forms

[Name of ASI program/agency] CONSENTIMIENTO PARA EVALUACIÓN

DEL PROGRAMA

El programa de actividades para despues de la escuela, en el cual participa su hijo(a), está fundado en parte

por The Colorado Trust. Durante del año, [name of ASI program/agency], junto con “The Colorado Trust”

y NRC (National Research Center, Inc.), evaluarán periódicamente el programa [name of ASI program].

Esperamos que la información obtenida durante la evaluación nos ayude a mejorar nuestro sistema al

demostrarnos la efectividad del programa [name of ASI program], y nos indique si quienes participan en él

están logrando desarrollar habilidades positivas para la juventud. A causa de que la evaluación del programa

incluye recoger respuestas de encuestas y desarollar conocimiento, dicha evaluación requiere investigación.

Como parte del proceso de la investigación, se le pedirá a su hijo(a) que complete un breve cuestionario.

Todo el material recogido como producto de esta evaluación será entregado a The Colorado Trust y al

National Research Center, Inc., para evaluar y analizar.

Preguntas acerca de las opinones de su hijo(a) sobre el programa y sus sentimientos y actividades personales

están incluidas en la encuesta. Las preguntas se centrarán en el nivel de satisfacción de su hijo(a) con

respecto al programa, y si se siente seguro(a) mientras participa en él y cómo el personal del programa ha

respondido a él(ella). También, dependiendo el tipo de programa en él cuál participa su hijo(a), se le hará

preguntas adicionales acerca del desarrollo positivo de jóvenes.

Hay ejemplos de preguntas en la lista siguiente:

• Me siento seguro(a) en este programa (Sí, Más o menos, No)

• El personal del programa me escucha (Sí, Más o menos, No)

• El programa me ayudó a sentirme mejor conmigo mismo (Sí, Más o menos, No)

• Este programa me ayudó a pasar más tiempo leyendo (Sí, Más o menos, No)

[ASI program staff may delete some or all of the questions below if they will not be asking questions related

to such topics]

• Al participar en este programa aprendí a evitar el tabaco (Sí, Más o menos, No)

• Al participar en este programa aprendí a evitar las drogas (Sí, Más o menos, No)

• Al participar en este programa aprendí a evitar el alcohol (Sí, Más o menos, No)

• Al participar en este programa aprendí a evitar las relaciones sexuales (Sí, Más o menos, No)

La participación de su hijo(a) en esta invgestigación es totalmente voluntaria, es decir, su hijo(a) tiene el

derecho de negarse a participar en ella. Su hijo(a) puede brincarse (no contestar) preguntas que no quiera

contestar. La participación en esta evaluación no es requerida y no afectará , ni positivamente ni

negativamente, su asistencia en el programa. También, si su hijo(a) sale del programa o está ausente

(incluyendo retiro voluntario, ausencia temporal, suspención, explusión, o terminación) del [name of ASI

program] (ya sea porque se trate de una ausencia transitoria, porque se retiró voluntariamente, o bien porque

fue suspendido o expulsado), igualmente podrá participar en esta evaluación.

La participación en esta evaluación podría implicar ciertos riesgos menores para su hijo(a). Es posible que

algunos de los jóvenes participantes se sientan avergonzados o incómodos con ciertas preguntas. Por

ejemplo, es posible que un joven no quiera responder a una pregunta sobre su autoestima o costumbres

personales. Como dicho anteriormente, su hijo(a) tiene el derecho de brincarse (no contestar) cualquiera de

las preguntas que le hagan sentirse avergonzado(a) o incómodo(a). Esta evaluación ha sido diseñada

especialmente para jóvenes e incluye el tipo de pregunta que encontrarían en la escuela u otro tipo de

programa. Sin embargo, si usted tiene cualquier duda, favor de ponerse en contacto con el personal del

programa para revisar el material de la evaluación.

Toda la información entregada por su hijo(a) para la evaluación se usará para propósitos estadísticos o de

evaluación, para que [Choose either: (su identidad no será divulgada. El nombre de su hijo(a) no aparecerá

en ninguno de los informes elaborados por [name of ASI program/agency], el NRC o The Colorado Trust.

Toda la información de su hijo(a) para la evaluación será confidencial.) (OR) (Toda la información de su

hijo(a) para la evaluación será confidencial.)] Sin embargo, hay dos excepciones a la promesa de

confidencialidad. La ley exige que en caso de que la información entregada revele la intención de suicidio,

homicidio, abuso o descuido de menores de edad, dicha información sea entregada a las autoridades

pertinentes. Además, si la información entregada para la evaluación formara parte de una órden judicial o

citación, [name of ASI program/agency] estaría obligado por ley a divulgar dicha información en virtud de la

citación correspondiente.

Toolkit for Evaluating Positive Youth Development 47

Toolkit for Evaluating Positive Youth Development 48

[If applicable] Además, es posible que el equipo de la evaluación del [name of ASI program/agency] quiera

examinar los archivos escolares (boletín de notas, calificaciones CSAP, calificaciones en pruebas regulares,

informes de asistencia) de los participantes en el programa. Su firma da su permiso a NRC y al personal del

[name of ASI program/agency] para obtener acceso a estos archivos.

Si tiene preguntas acerca de la evaluación o los derechos de su hijo(a) como participante el la evaluación, o si

quiere obtener una copia de la encuesta completa antes de firmar el consentimiento, favor de ponerse en

contacto con [ASI program staff name] al [phone number] ___________________________________.

He leido y comprendido la descripción de la evaluación incluída en este documento. En cuanto a los puntos

que no comprendí, solicité y recibí una explicación satisfactoria. He dado mi permiso a que mi hijo(a)

participe en esta evaluación hasta el fin del mes de Mayo 2005, el final de The Colorado Trust’s After-

School Initiative, y entiendo que tengo el derecho de retirar mi consentimiento en cualquier momento sin

perder ni terminar la participación de mi hijo(a) en [name of ASI program/agency]. También, entiendo que

la privacidad de mi hijo(a) estará protegida [Choose either: (porque sus respuestas no se pueden rastrear a

él(ella).) OR (porque sus respuestas no se reportarán individualmente.)]

Nombre del joven participante: (en letra de molde) ____________________________________________

Nombre del padre/custodio: (en letra de molde) ______________________________________________

Firma del padre/custodio: _______________________________________________________________

(si el menor tiene dos custodios, recomendamos incluir la firma de ambos)

Nombre del custodio: (en letra de molde) ___________________________________________________

Firma del custodio: ____________________________________________________________________

Fecha: ______________________________________________________________________________

Asentimiento de jovenes para participar en la evaluación del programa de [name of

ASI program/agency]

Como participante en [name of ASI program/agency], estoy de acuerdo en formar parte de un estudio,

contestaré preguntas acerca de mis actividades, sentimientos y opiniones sobre [name of ASI program] y

cómo mi participación en el programa me ha ayudado o no me ha ayudado. También comprendo que mis

respuestas son privadas [Choose either: (y que mi nombre no está en la hoja de respuestas.) (OR) (y que mis

respuestas se reportarán como parte de un grupo, y no individualmente.)] Mis respuestas no estarán sujetas a

calificaciones y nadie las discutirá conmigo a menos que yo lo solicite. Estoy de acuerdo en participar en esta

evaluación hasta el fin del mes de mayo 2005. Puedo decidir de no participar mas en la evaluación en

cualquier momento, sin causarme ningún problema. En caso de que decida no contestar las preguntas,

podre seguir asistiendo al programa de [name of ASI program].

Nombre del niño: (en letra de molde) ______________________________________________________

Firma del niño: _______________________________________________________________________

Fecha: ______________________________________________________________________________

Fecha de nacimiento: ___________________________________________________________________

Testigo adulto*:__________________________________ Fecha: _______________________________

Firma del testigo adulto: ________________________________________________________________

*El testigo adulto tiene que estar presente durante del proceso de asentimiento.

[Program copy OR Child Participant copy]

Toolkit for Evaluating Positive Youth Development 49

Toolkit for Evaluating Positive Youth Development 50

Appendix V. English and Spanish Comparison of Question Sets

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

Academic

Success

Coming to [this program] has helped me to

…

El participar en [este programa] me ha

ayudado a …

Improve my grades in school Mejorar mis calificaciones en la escuela Grades

Improve my grade in language arts

(reading/writing)

Mejorar mi calificación de lenguaje

(lectura/escritura)

Grades

Improve my grade in math Mejorar mi calificación en matemáticas Grades

Improve my grade in science Mejorar mi calificación en ciencia Grades

Do better in school Tener más éxito en mis estudios Frequency

Do better in language arts (reading/writing) Tener más éxito en lenguaje (lectura/escritura) Frequency

Do better in math Tener más éxito en matemáticas Frequency

Do better in science Tener más éxito en ciencia Frequency

Try harder in school Esforzarme más en mis estudios Frequency

Spend more time doing my homework Dedicarle más tiempo a mi tarea Hours

Participate more in class activities Participar más en actividades escolares Frequency

Become more interested in going to school Aumentar mi interés en ir a la escuela Frequency

Care more about my school Darle más importancia a mis estudios Frequency

Get along better with my classmates Llevarme mejor con mis compañeros de clase Frequency

Get along better with my teachers Llevarme mejor con mis maestros Frequency

Spend more time reading for fun (not for

school)

Pasar más tiempo leyendo por diversión, y no

solamente por obligación escolar

Hours

Arts and

Recreation

Because I came to [this program] … Por haber participado en [este programa] ...

I spend more time doing arts and crafts Dedico más tiempo a la artesanía Hours

I enjoy arts and crafts more Me gustan más las artesanías que antes Frequency

I am better in arts and crafts Me desarrollo mejor en artesanía Frequency

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

I spend more time in music, dance and/or

theater

Ahora participo más en música, baile y/o teatro Hours

I enjoy music, dance and/or theater more Ahora me gusta más el teatro, baile y/o la

música

Frequency

I am better at music, dance and/or theater Me desarrollo mejor en el teatro, baile y/o la

música

Frequency

I spend more time doing sports, exercise

and/or recreation activities

Ahora participo más en deportes, ejercicio y/o

actividades recreativas

Hours

I enjoy sports, exercise and/or recreation

activities more

Ahora me gustan más los deportes, el ejercicio

y/o actividades recreativas

Frequency

I am better at sports, exercise and/or recreation

activities

Me desarrollo mejor en deportes, ejercicio y/o

actividades recreativas

Frequency

Community

Involvement

Coming to [this program] has helped me …

El participar en [este programa] me ha

ayudado a …

Spend more time volunteering or helping

others in my community

Pasar más tiempo como voluntario ayudando a

gente de mi comunidad

Hours

Spend more time looking for opportunities to

help others in my community

Pasar más tiempo buscando la oportunidad de

ayudar a gente de mi comunidad

Hours

Feel more important to my community Sentirme más significativo dentro de mi

comunidad

Frequency

Feel a stronger connection to my community Sentir una conexión más grande con mi

comunidad

Frequency

Feel better about myself because I help others Sentirme mejor de mí mismo por ayudar a la

gente

Frequency

Toolkit for Evaluating Positive Youth Development 51

Toolkit for Evaluating Positive Youth Development 52

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

Cultural

Competency

Because I came to [this program] …

Por haber participado en [este programa] ...

I know more about the good things that people

who look or sound different than me have

done (for example, African Americans, Mexican

Americans, Native Americans, Asian Americans

and others)

Tengo más conocimiento de las cosas buenas

que se han logrado por personas quienes tienen

un semblante diferente al mío (por ejemplo;

Afro-Americanos, México-Americanos, Nativo-

Americanos, Asia-Americanos, Etc.)

Frequency

I talk more with young people who look or

sound different than me

Converso más con los jóvenes quienes tienen

un semblante diferente al mío

Frequency

I feel more comfortable being around young

people who look or sound different than me

Siento más confianza al estar con jóvenes

quienes tienen un semblante diferente al mío

Frequency

I understand that someone who looks or

sounds different than me may not be that

different after all

Comprendo que alguien quien tiene un

semblante diferente al mío, tal vez al fin de

cuenta, no sea tan diferente

Frequency

Being involved in [this program] has

helped me to …

El involucrarme en [este programa] me ha

ayudado a...

Know more about people of other cultures,

races or ethnic groups.

Tener más conocimiento de otras culturas,

razas o grupos étnicos

Amount

Care more about young people of other

cultures, races or ethnic groups than me

Preocuparme más por el bienestar de jóvenes

de otras culturas, razas o grupos étnicos

Frequency

Have more respect for young people of other

cultures, races or ethnic groups than me

Tener más respeto para los jóvenes de otras

culturas, razas o grupos étnicos

Frequency

Feel more comfortable with young people of

other cultures, races or ethnic groups

Sentir más confianza con los jóvenes de otras

culturas, razas o grupos étnicos

Frequency

Talk more to young people who speak

languages other than English

at home

Conversar más con los jóvenes quienes no

hablan inglés en su hogar

Frequency

Try harder not to judge people based on skin

color

Ser más consciente de no juzgar a personas por

el color de su piel

Frequency

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

Be more aware that I sometimes have ideas

about people that are not true (Incorrect or

mistaken)

Ser más consciente de que malinterpreto las

ideas que formo de otras personas y que no

siempre son ciertas, correctas o exactas

Frequency

Because I came to [this program] … Por haber participado en [este programa] ...

I know more about my own culture, race or

ethnic group

Tengo más conocimiento de mi propia cultura,

raza o grupo étnico

Amount

I am more interested in my own culture, race

or ethnic group

Me interesa más mi propia cultura, raza o grupo

étnico

Frequency

I spend more time celebrating my own

culture, race or ethnic group

Paso más tiempo celebrando mi propia cultura,

raza o grupo étnico

Hours

I feel more connected to my own culture,

race or ethnic group

Siento una conexión más grande con mi propia

cultura, raza o grupo étnico

Frequency

I feel more pride for my own culture, race or

ethnic group

Siento más orgullo de mi propia cultura, raza o

grupo étnico

Frequency

Life Skills

Because I came to [this program] … Por haber participado en [este programa] ...

I get along better with other people my

age

Me comporto mejor con otras personas de mi

misma edad

Frequency

I am better at making friends Me desarrollo mejor al formar amistades Frequency

I am better at telling others about my

ideas and feelings

Me desarrollo mejor al compartir mis ideas y

sentimientos con los demás

Frequency

I am better at listening to other people Es más fácil entender a otras personas Frequency

I work better with others on a team Es más fácil trabajar en equipo con otros Frequency

I make better decisions Tomo mejores decisiones Frequency

I am better at planning ahead Me desarrollo mejor al planear por adelantado Frequency

I am better at setting goals Me desarrollo mejor al establecer metas

personales

Frequency

I am better at solving problems Me desarrollo mejor al solucionar problemas Frequency

I am more of a leader Me desarrollo mejor como líder Frequency

I am better at taking care of problems

without violence or fighting

Me desarrollo mejor al solucionar problemas

sin violencia ni enfrentamientos

Frequency

Toolkit for Evaluating Positive Youth Development 53

Toolkit for Evaluating Positive Youth Development 54

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

Positive Core

Values

Because I came to [this program] … Por haber participado en [este programa] ...

I care more about other people Me importa más el bienestar de los demás Frequency

I care more about the feelings of other people Me preocupan más los sentimientos de otras

personas

Frequency

I am better at standing up for what I believe Me desarrollo mejor al defender mis creencias Frequency

I tell the truth more often even when it is hard Digo la verdad con más frecuencia, aunque sea

difícil hacerlo

Frequency

I am better at taking responsibility for my

actions

Puedo aceptar responsabilidad por mis propias

acciones con más facilidad

Frequency

I am more interested in community and world

problems

Me interesan más los problemas de la

comunidad y del mundo

Frequency

I am better at speaking up for people who have

been treated unfairly

Es más fácil defender a las personas que son

tratadas con injusticia

Frequency

Positive Life

Choices

Being involved in [this program] has

helped me to …

El involucrarme en [este programa] me ha

ayudado a...

Better say “no” to things I know are wrong Decir “no” a las actividades que son malas Frequency

Stay out of trouble Evitar problemas Frequency

Stay away from tobacco Evitar el tabaco Frequency

Stay away from alcohol Evitar el alcohol Frequency

Stay away from drugs Evitar las drogas Frequency

Stay away from sex Evitar relaciones sexuales Frequency

Stay away from violence and fighting Evitar la violencia y el pelear Frequency

Outcome

Domain

English Post-only Question Spanish Post-only Question Scale

Sense of Self

Coming to [this program] has helped me

to …

El participar en [este programa] me ha

ayudado a …

Feel better about myself Sentirme mejor de mí mismo Frequency

Feel I have more control over things that

happen to me

Sentirme más en control de lo que me pasa Frequency

Feel that I can make more of a difference Sentir que puedo hacer una diferencia Frequency

Learn I can do things I didn’t think I could do

before

Entender que tengo habilidades que no conocía Frequency

Feel better about my future Sentirme mejor sobre mi futuro Frequency

Feel I am better at handling whatever comes

my way

Tener más confianza para enfrentar el futuro Frequency

Toolkit for Evaluating Positive Youth Development 55

Appendix VI. Protocol Worksheet for ASI Toolkit Survey Administration

Instructions: Complete a separate protocol worksheet for each activity or group of surveys administered and submit to

your evaluation liaison no later than two weeks before the survey administration date. Some grantees may only fill out one

worksheet for all program activities offered.

Background Information:

a) Grantee name:_______________________ b) Contact name:_______________________

c) Contact phone:______________________ d) Contact e-mail:______________________

e) Activity name:_____________________________________________________

Question Response

1) Which youth participants are eligible

to take the survey?

Eligible youth should include all registered

participants in 4th –9th grade unless a sampling

plan is developed or you and your evaluation liaison

agree on another standard. Consult with your

evaluation liaison.

2) How many youth will you ask to take

the survey?

This should equal the number of eligible

participants minus the number of youth for which

you do NOT have consent forms.

3) Will the survey be administered in

Spanish as well as English?

 Yes  No

4) When will data be collected?

More than one date may need to be scheduled if it

will help improve the response rate. Consider

attendance on each day of the week and whether the

same youth attend on multiple days.

Provide an adequate amount of time for completion

so that youth do not feel rushed or like they are

missing out on other program activities.

1st Date:

Time: From ________ to ________

2nd Date:

Time: From ________ to ________

3rd Date:

Time: From ________ to ________

5) Who will serve as the survey

administrator?

Name: _______________________________

6) Who will serve as the Spanish-

speaking survey administrator?

If needed.

Name: _______________________________

Toolkit for Evaluating Positive Youth Development 56

7) Who will assist the survey

administrator(s) with the survey

administration?

Name: _______________________________

8) Has the survey administrator(s)

reviewed the Survey Administrator

Guidelines and Procedures?

 Yes  No

If no, when will this occur? ______________

9) What explanatory text will be recited

when distributing surveys to youth?

Attach text to this worksheet.

10) Who is responsible for gathering

supplies?

Pens/pencils, copies of surveys, etc.?

Name: _______________________________

11) Do you have a box or something else

in which participants will place their

completed surveys?

 Yes  No

Describe: _____________________________

12) For programs with youth attending

multiple activities, how will you make

sure that youth are only surveyed one

time? Please explain.

Consult with your evaluation liaison to develop a

strategy if needed.

13) Describe how the survey

administration will take place on the

administration day.

Use the back of this form or a separate

sheet if needed.

Example: Kids will enter the lunchroom at 3:30

and have their snack for 15 minutes. Then Ester

will ask the youth to sit quietly and listen to the

survey instructions. Joey will pass out the surveys

and pencils and have youth find a quiet place to fill

out the survey by them Youth will have as much

time as they need to complete the survey. As

students finish the survey, they will bring it up to

the front of the room, place it face down in the

survey box and then join Robby outside for games.

Toolkit for Evaluating Positive Youth Development 57

ool

kit f

atin

itiv

th D

elo

Appendix VII. Toolkit Outcome Domains Linked to Positive Youth Development Models

TABLE 1 - Toolkit Outcome Domains Linked to Positive Youth Development Models

Note: Many 8 Habits of the Heart outcomes are placed in the internal assets category although the framework suggests that the traits are

demonstrated first by others and then internalized by the youth

Toolkit

Outcome

Domains

Toolkit Outcome Indicators

Search Institute

Outcomes

4-H 4 Fold Model

8 Habits of

the Heart

Academic

Success

Grades

Skills: math, science, reading, writing

CSAP scores

School attachment

School engagement

Interest in learning (e.g., reading for

pleasure)

Graduation rates

Commitment to learning

Achievement motivation

School engagement

Homework

Bonding to school

Reading for pleasure

Head Skills:

Utilizing scientific

method

Processing

information

Understanding

systems

Learning to learn

Visualizing

information

Reasoning

Hand Skills:

Completing a

project/task

Arts and

Recreation

Appreciation for the arts (and crafts),

music, dance, or theater

Performance in arts (and crafts), music,

dance or theater

Performance in sports, exercise and

recreation activities

Constructive use of time

Creative activities

Youth programs

Hand Skills:

Practicing creativity

Toolkit

Outcome

Domains

Toolkit Outcome Indicators

Search Institute

Outcomes

4-H 4 Fold Model

8 Habits of

the Heart

Time spent in arts (and crafts), music,

dance or theater

Time spent in sports, exercise and

recreation activities

Community

Involvement

Time spent in community service

Sense of importance to community (youth

as resources)

Self concept due to community

involvement

Empowerment

Community values youth

Youth as resources

Service to others

Hand Skills:

Learning through

community service

Responsible

citizenship

Service learning

Volunteering

Cultural

Competency

Respect for others

Comfort with others

Knowledge of other cultures, races or

ethnic groups

Respect for other cultures, races or ethnic

groups

Comfort with other cultures, races or

ethnic groups

Knowledge of own culture

Interest in own culture

Sense of belonging to cultural group

Pride in one’s culture

Social competencies

Cultural competency

Heart Skills:

Valuing diversity

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

Toolkit

Outcome

Domains

Toolkit Outcome Indicators

Search Institute

Outcomes

4-H 4 Fold Model

8 Habits of

the Heart

Life Skills

Friendship skills

Communication skills

Decisionmaking skills

Planning for the future

Leadership skills

Goal-setting skills

Problem-solving skills

Conflict-resolution skills

Teamwork

Social competencies

Planning and decision

making

Interpersonal competence

Peaceful conflict resolution

Head skills:

Making decisions

Solving problems

Reasoning

Thinking critically

Planning and

organizing

Heart Skills:

Communicating

Interacting socially

Resolving conflicts

Building relationships

Hand skills:

Working in a team

Exercising leadership

Health Skills:

Managing stress

Brotherhood

Responsibility

Friendship

Dependability

Positive Life

Choices

Resistance skills

Restraint from drug abuse, tobacco use,

alcohol use, violent behaviors and early

sexual activity

Social competencies

Resistance skills

Health Skills:

Utilizing resistance

skills

Being resilient

Toolkit

Outcome

Domains

Toolkit Outcome Indicators

Search Institute

Outcomes

4-H 4 Fold Model

8 Habits of

the Heart

Positive Core

Values

Caring

Empathy

Integrity

Honesty

Responsibility

Equality and fairness

Positive values

Caring

Equality and social justice

Integrity

Honesty

Responsibility

Heart Skills:

Sharing

Valuing social justice

Caring for others

Being empathetic

Health Skills:

Being responsible

Practicing integrity

Courage

Sense of Self

Self concept (self-confidence, self-esteem,

self-worth)

Empowerment

Positive outlook

Sense of purpose

Positive identity

Personal power

Self-esteem

Sense of purpose

Positive view of personal

future

Head Skills:

Achieving goals

Health Skills:

Developing self-

esteem

Managing yourself

Developing a sense of

purpose

Developing a positive

view of future

Hope

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

TABLE 2 - Toolkit Outcome Domains Linked to Positive Youth Development Models

Note: Bernard’s Resilience Assessment is used by the California Healthy Kids Survey. It is based on Bonnie Bernard’s resiliency framework.

Toolkit Domains Toolkit Outcome Indicators

Bernard’s Resilience

Assessment

Character Counts

Communities

of Caring

Academic

Success

Grades

Skills: math, science, reading, writing, CSAP

scores

School attachment

School engagement

Interest in learning (e.g., reading for pleasure)

Graduation rates

Improved

academic

performance

Arts and

Recreation

Appreciation for the arts (and crafts), music,

dance or theater

Performance in arts (and crafts), music, dance or

theater

Performance in sports, exercise and recreation

activities

Time spent in arts (and crafts), music, dance or

theater

Time spent in sports, exercise and recreation

activities

Community

Involvement

Time spent in community service

Sense of importance to community (youth as

resources)

Self concept due to community involvement

Meaningful

participation at home,

in school and in

community/neighbor-

hood

Toolkit Domains Toolkit Outcome Indicators

Bernard’s Resilience

Assessment

Character Counts

Communities

of Caring

Cultural

Competency

Respect for others

Comfort with others

Knowledge of other cultures, races or ethnic

groups

Respect for other cultures, races or ethnic groups

Comfort with other cultures, races or ethnic

groups

Knowledge of own culture

Interest in own culture

Sense of belonging to cultural group

Time spent learning about own culture

Sense of belonging to cultural group

Pride in one’s culture

Social competence

Respect for diversity

Life Skills

Friendship skills

Communication skills

Decision making skills

Planning for the future

Leadership skills

Goal-setting skills

Problem-solving skills

Conflict-resolution skills

Teamwork

Social competence

Cooperation and

communication skills

Problem-solving skills

Decision-

making skills

Positive Life

Choices

Resistance skills

Restraint from drug abuse, tobacco use, alcohol

use, violent behaviors, early sexual activity

Decrease in

risk behaviors

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

Toolkit Domains Toolkit Outcome Indicators

Bernard’s Resilience

Assessment

Character Counts

Communities

of Caring

Positive Core

Values

Caring

Empathy

Integrity

Honesty

Responsibility

Equality and fairness

Social competence

Empathy

Trustworthiness

Respect

Responsibility

Fairness

Caring

Responsibility

Commitment

to values

Encourage

caring

Sense of Self

Self-concept (self-confidence, self-esteem, self-

worth)

Empowerment

Positive outlook

Sense of purpose

Autonomy and sense

of self

Personal conviction,

self-efficacy, internal

locus of control and

self awareness

Sense of meaning and

purpose

Optimism, goals and

aspirations,

achievement and

motivation

Build

confidence

Increase

feelings of

self-worth

TABLE 3 – Toolkit Outcome Domains Linked to Positive Youth Development Models

Toolkit

Domains

Toolkit Outcome Indicators

Communities

That Care

Henderson and

Milstien’s Resiliency

Model

Individual

Risk and

Protective

Factors

Index

Academic

Success

Grades

Skills: math, science, reading, writing, CSAP scores

School attachment

School engagement

Interest in learning (e.g., reading for pleasure)

Graduation rates

Decrease truancy

Decrease drop-out

rates

Increase school

engagement and

performance

Love of learning

School

bonding

Arts and

Recreation

Appreciation for the arts (and crafts), music, dance,

or theater

Performance in arts (and crafts), music, dance or

theater

Performance in sports, exercise and recreation

activities

Time spent in arts (and crafts), music, dance or

theater

Time spent in sports, exercise and recreation

activities

Community

Involvement

Time spent in community service

Sense of importance to community (youth as

resources)

Self-concept due to community involvement

Service

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

Toolkit

Domains

Toolkit Outcome Indicators

Communities

That Care

Henderson and

Milstien’s Resiliency

Model

Individual

Risk and

Protective

Factors

Index

Cultural

Competency

Respect for others

Comfort with others

Knowledge of other cultures, races or ethnic groups

Respect for other cultures, races or ethnic groups

Comfort with other cultures, races or ethnic groups

Knowledge of own culture

Interest in own culture

Sense of belonging to cultural group

Time spent learning about own culture

Sense of belonging to cultural group

Pride in one’s culture

Life Skills

Friendship skills

Communication skills

Decisionmaking skills

Planning for the future

Leadership skills

Goal-setting skills

Problem-solving skills

Conflict resolution skills

Teamwork

Increase positive

peer experiences

Relationships (e.g.,

sociability, ability to be

a friend and form

positive relationships)

Life skills (e.g.,

decisionmaking,

assertiveness)

Self-control

Assertiveness

Cooperation

Positive Life

Choices

Resistance skills

Restraint from drug abuse, tobacco use, alcohol use,

violent behaviors and early sexual activity

Life skills (impulse

control)

Pro-social

norms

Toolkit

Domains

Toolkit Outcome Indicators

Communities

That Care

Henderson and

Milstien’s Resiliency

Model

Individual

Risk and

Protective

Factors

Index

Positive

Core

Values

Caring

Empathy

Integrity

Honesty

Responsibility

Equality and fairness

Sense of

Self

Self-concept (self-confidence, self-esteem, self-worth)

Empowerment

Positive outlook

Sense of purpose

Self-worth (and self-

confidence)

Positive view of

personal future

Self-concept

Self-efficacy

Positive

outlook

Confidence

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

Table 4 - Toolkit Questions Categorized by Outcome

Outcome Domain Advanced Question

Academic Success

Coming to [this program] has helped me

to…

Improve my grades in school

Improve my grade in language arts (reading/writing)

Improve my grade in math

Improve my grade in science

Do better in school

Do better in language arts (reading/writing)

Do better in math

Do better in science

Coming to [this program] has helped me

to…

Try harder in school

Spend more time doing my homework

Participate more in class activities

Become more interested in going to school

Care more about my school

Get along better with my classmates

Get along better with my teachers

Spend more time reading for fun (not for school)

Arts and Recreation

Because I came to [this program] … I spend more time doing arts and crafts

I enjoy arts and crafts more

I am better in arts and crafts

I spend more time in music, dance and/or theater

I enjoy music, dance and/or theater more

I am better at music, dance and/or theater

I spend more time doing sports, exercise and/or recreation

Outcome Domain Advanced Question

activities

I enjoy sports, exercise and/or recreation activities more

I am better at sports, exercise and/or recreation activities

Community Involvement

Coming to [this program] has helped

me…

Spend more time volunteering or helping others in my

community

Spend more time looking for opportunities to help others in my

community

Feel more important to my community

Feel a stronger connection to my community

Feel better about myself because I help others

Cultural Competency

Because I came to [this program]… I know more about the good things that people who look or

sound different than me have done (like African Americans,

Mexican Americans, Native Americans, Asian Americans and

others)

I talk more with young people who look or sound different than

I feel more comfortable being around young people who look or

sound different than me

I understand that someone who looks or sounds different than

me may not be that different after all

Being involved in [this program] has

helped me to …

Know more about people of other cultures, races or ethnic

groups

Care more about young people of other cultures, races or ethnic

groups than me

Have more respect for young people of other cultures, races or

ethnic groups than me

Feel more comfortable with young people of other cultures, races

or ethnic groups

ool

kit f

atin

itiv

th D

elo

ool

kit f

atin

itiv

th D

elo

nt 7

Outcome Domain Advanced Question

Talk more to young people who speak languages other than

English at home

Try harder not to judge people based on skin color

Be more aware that I sometimes have ideas about people that are

not true (Incorrect or mistaken)

Because I came to [this program]…

I know more about my own culture, race or ethnic group

I am more interested in my own culture, race or ethnic group

I feel more connected to my family’s cultural traditions

I feel more connected to my own culture, race or ethnic group

I feel more pride for my own culture, race or ethnic group

I have more respect for members of my own community

Life Skills

Because I came to [this program]… I get along better with other people my age

I am better at making friends

I am better at telling others about my ideas and feelings

I am better at listening to other people

I work better with others on a team

I make better decisions

I am better at planning ahead

I am better at setting goals

I am better at solving problems

I am more of a leader

I am better at taking care of problems without violence or fighting

Positive Core Values

Because I came to [this program]… I care more about other people

I care more about the feelings of other people

I am better at standing up for what I believe

I tell the truth more often even when it is hard

I am better at taking responsibility for my actions

Outcome Domain Advanced Question

I am more interested in community and world problems

I am better at speaking up for people who have been treated

unfairly

Positive Life Choices

Being involved in [this program] has helped

me to…

Better say “no” to things I know are wrong

Stay out of trouble

Stay away from tobacco

Stay away from alcohol

Stay away from drugs

Stay away from sex

Stay away from violence and fighting

Sense of Self

Coming to [this program] has helped me

to…

Feel better about myself

Feel I have more control over things that happen to me

Feel that I can make more of a difference

Learn I can do things I didn’t think I could do before

Feel better about my future

Feel I am better at handling whatever comes my way

ool

kit f

atin

itiv

th D

elo

nt 71

ool

kit f

atin

itiv

th D

elo

nt 7

Program Quality Question

Please check the box below that is closest to

how you feel about [this program].

There are interesting activities at [this program]

The activities are fun at [this program]

I learn about new things at [this program]

I like coming to [this program]

There are rules I am expected to follow at [this program]

I feel safe at [this program]

I feel like people are happy to see me here

I tell my friends to come to [this program]

Please check the box below that is closest to

how you feel about the staff at [this program].

Staff cares about me

Staff helps me feel important

Staff gives me lots of support

Staff spends time with me one-on-one

Staff can be trusted

I like the staff here

Staff expects me to try hard and do my best

Staff tells me when I do a good job

I could go to a staff member at [program] for advice if I had a

serious problem

Staff listens to what I have to say

Staff asks me to plan, choose or lead activities

Staff encourages me to do volunteer work

Staff treats all kids fairly

Open-Ended Questions What do you like best about coming to [this program]?

If you could change one thing about [this program] what would it

be?

Background Information Advanced Question

Attendance

How often do you come to [this project]?

(Every day or almost every day [4-5 times per week], 2 to 3

times a week, once a week, a couple times a month, less than

once a month)

When did you start coming to [this project]?

(This fall, over the summer, last spring, last year or longer ago

than that)

When did you start coming to [this project]?

(Within the last 3 months, 4 to 6 months ago, 7 to 12 months

ago, more than 1 year ago)

Age

How old are you?

(6 years or younger, 7 years, 8 years, 9 years, 10 years, 11 years,

12 years, 13 years, 14 years, 15 years or older)

Grade

What grade are you in?

Sex/Gender

Are you a girl/female or a boy/male?

(Girl/female, boy/male)

Race/Ethnicity

Which best describes you?

(Caucasian/white, African American/black, Asian/ Pacific

Islander, Latino/Hispanic, Native American, bi-racial/ multi-

racial, other)

Activity Identification

Check each of the after-school activities you participate in

at [THIS PROGRAM].

ool

kit f

atin

itiv

th D

elo

nt 7

ool

kit f

atin

itiv

th D

elo

nt 74

Appendix VIII. Academic Success Tracking Forms

There are 4 tracking forms included in this toolkit. These forms are to be completed using information provided to a program from the

schools that youth participants attend. The tracking forms are to be used in a confidential manner. Since survey question sets are

administered anonymously, program staff will not be able to link survey results with individual participant Colorado Student Assessment

Program (CSAP) scores, grades or attendance. However, the tracking forms will provide interesting secondary data that may be useful for

program improvement. Each form is described below. ASI program staff can obtain an electronic version of these forms in Microsoft

Excel from their NRC evaluation liaisons.

CSAP: Colorado Student Assessment Program is administered to youth in grades 3 through 10. Each grade level completes from 2 to 4

subjects — reading, writing, math and science. The CSAP tracking form should be completed using student scores prior to program

attendance and then a second time, either during or after participation in the program. Change in proficiency will be computed by

comparing the second score to the first score.

GRADES: Schools may use several grading scales (e.g., satisfactory-unsatisfactory; A, B, C, D, F; excellent, very good, satisfactory, needs

improvement) and semester or trimester grading. The tracking form may need to be adjusted to accommodate for a school’s grading

system as well as the subject areas being tracked. Similar to the CSAP scores, grades should be entered for students prior to program

attendance and then again, either during or after participation in the program.

ATTENDANCE: An attendance tracking form is provided so the number of days a program participant is absent can be transferred from

school records. The number of days absent in the semester or year is recorded prior to the program and then again after participation in the

program.

GRADUATION/DROP-OUT: A leading indicator of school success is the completion of school. “Graduation” for toolkit purposes can

be graduation from high school, middle school or elementary school. Program staff may also use the form to record the number of youth

participants who stay in school rather than drop out.

CSAP Tracking Form

Reading

1 = proficient

2 = partially proficient

3 = not proficient

Writing

1 = proficient

2 = partially proficient

3 = not proficient

Math

1 = proficient

2 = partially proficient

3 = not proficient

Science

1 = proficient

2 = partially proficient

3 = not proficient

Student ID

Before

Program

After

Program

Before

Program

After

Program

Before

Program

After

Program

Before

Program

After

Program

*Note: Program staff may also enter data on a computer into a Microsoft Excel spreadsheet.

ool

kit f

atin

itiv

th D

elo

nt 7

ool

kit f

atin

itiv

th D

elo

nt 7

Grade Tracking Form

Overall GPA

Reading Writing Math Science

Student ID

Before

Program

After

Program

Before

Program

After

Program

Before

Program

After

Program

Before

Program

After

Program

Before

Program

After

Program

*Note: Program staff can also enter data on a computer into a Microsoft Excel spreadsheet.

Attendance Tracking Form

Number of Days Absent

Student ID

Before Program After Program

*Note: Program staff can also enter data on a computer into a Microsoft Excel spreadsheet.

ool

kit f

atin

itiv

th D

elo

nt 77

ool

kit f

atin

itiv

th D

elo

nt 7

Graduation/Drop-Out Tracking Form

School Status (Place an X in the applicable category)

Student ID

Graduated In school (eligible but did

not graduate on time)

In school (not eligible –

too young)

Dropped out before

graduating

*Note: Program staff can also enter data on a computer into a Microsoft Excel spreadsheet.

Appendix IX.

Complete Post-only Toolkit

Survey Question Set

ool

kit f

atin

itiv

th D

elo

nt 7

[NAME OF ASI PROGRAM/AGENCY]

YOUTH PARTICIPANT SURVEY

Tell us what you think!

This survey is being used to get your ideas about [this program]. It is not a test and there are no

right or wrong answers. Please take your time and be sure to answer each question based on

what you really think. Please do NOT put your name on this survey.

Your answers are completely private.

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

1) Please check the box below that is closest to how you feel about [this program].

Yes Kind of Not Really

a) There are interesting activities at [this program]   

b) The activities are fun at [this program]   

c) I learn about new things at [this program]   

d) I like coming to [this program]   

e) There are rules I am expected to follow at [this program]   

f) I feel safe at [this program]   

g) I feel like people are happy to see me here   

h) I tell my friends to come to [this program]   

2) Please check the box below that is closest to how you feel about [this program].

Yes Kind of Not Really

a) Staff care about me   

b) Staff help me feel important   

c) Staff give me lots of support   

d) Staff spend time with me one-on-one   

e) Staff can be trusted   

f) I like the staff here   

g) Staff expect me to try hard and do my best   

h) Staff tell me when I do a good job   

i) I could go to a staff member at [this program] for advice

if I had a serious problem   

j) Staff listen to what I have to say   

k) Staff ask me to plan, choose or lead activities   

l) Staff encourage me to do volunteer work   

m) Staff treat all kids fairly   

ool

kit f

atin

itiv

th D

elo

[Academic Success – For ASI grantees choosing this domain, required questions appear in

bold.]

3) Coming to the [this program] has helped me to …

Yes Kind of Not Really

a) Improve my grades in school   

b) Improve my grade in language arts (reading/writing)   

c) Improve my grade in math   

d) Improve my grade in science   

e) Do better in school   

f) Do better in language arts (reading/writing)   

g) Do better in math   

h) Do better in science   

i) Try harder in school   

j) Spend more time doing my homework   

k) Participate more in class activities   

l) Become more interested in going to school   

m) Care more about my school   

n) Get along better with my classmates   

o) Get along better with my teachers   

p) Spend more time reading for fun (not for school)   

[Arts & Recreation – For ASI grantees choosing this domain, question “sets” are required

separately: a-c, d-f or g-i.]

4) Because I came to [this program] …

Yes Kind of Not Really

a) I spend more time doing arts and crafts   

b) I enjoy arts and crafts more   

c) I am better in arts and crafts   

d) I spend more time in music, dance and/or theater   

e) I enjoy music, dance and/or theater more   

f) I am better at music, dance and/or theater   

g) I spend more time doing sports, exercise and/or

recreation activities   

h) I enjoy sports, exercise and/or recreation activities more   

i) I am better at sports and/or recreation activities   

[Community Involvement – For ASI grantees choosing this domain, all questions are required.

FOR ADVANCED ONLY.]

5) Coming to [this program] has helped me to …

Yes Kind of Not Really

a) Spend more time volunteering or helping others in

my community   

b) Spend more time looking for opportunities to help

others in my community   

c) Feel more important to my community   

d) Feel a stronger connection to my community   

e) Feel better about myself because I help others   

ool

kit f

atin

itiv

th D

elo

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

ADVANCED ONLY.]

6) Being involved in [this program] has helped me to…

Yes Kind of Not Really

a) Know more about people of other cultures, races

or ethnic groups   

b) Care more about young people of other cultures,

races or ethnic groups   

c) Have more respect for young people of other

cultures, races or ethnic groups   

d) Feel more comfortable with young people of

other cultures, races or ethnic groups   

e) Talk more to young people who speak languages

other than English at home   

f) Try harder not to judge people based on skin color   

g) Be more aware that I sometimes have ideas about

people that are not true (incorrect or mistaken)   

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

ADVANCED ONLY.]

6a) Because I came to [this program] …

Yes Kind of Not Really

a) I know more about my own culture, race or ethnic

group   

b) I am more interested in my own culture, race or

ethnic group   

c) I feel more connected to my family’s cultural traditions   

d) I feel more connected to my own culture, race or

ethnic group   

e) I feel more pride in my own culture, race or ethnic group   

f) I have more respect for members of my own community   

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

BASIC ONLY.]

7) Because I came to [this program] …

Yes Kind of Not Really

a) I know more about the good things that people who

look or sound different than me have done (like African

Americans, Mexican Americans, Native Americans,

Asian Americans and others)   

b) I talk more with young people who look or sound

different than me   

c) I feel more comfortable being around young people who

look or sound different than me   

d) I understand that someone who looks or sounds

different than me may not be that different after all   

ool

kit f

atin

itiv

th D

elo

[Life Skills – For ASI grantees choosing this domain, a, b, d, e and j are required FOR BASIC.

All questions are required FOR ADVANCED.]

8) Because I came to [this program]…

Yes Kind of Not Really

a) I get along better with other people my age   

b) I am better at making friends   

c) I am better at telling others about my ideas and feelings   

d) I am better at listening to other people   

e) I work better with others on a team   

f) I make better decisions   

g) I am better at planning ahead   

h) I am better at setting goals   

i) I am better at solving problems   

j) I am more of a leader   

k) I am better at taking care of problems without violence

or fighting   

[Positive Life Choices – For ASI grantees choosing this domain, a, b and g are required. FOR

ADVANCED ONLY.]

9) Coming to [this program] has helped me to…

Yes Kind of Not Really

a) Better say “no” to things I know are wrong   

b) Stay out of trouble   

c) Stay away from tobacco   

d) Stay away from alcohol   

e) Stay away from drugs   

f) Stay away from sex   

g) Stay away from violence and fighting   

[Positive Core Values – For ASI grantees choosing this domain, a, b, c, and d are required FOR

BASIC. All questions are required FOR ADVANCED.]

10) Because I came to [this program]…

Yes Kind of Not Really

a) I care more about other people   

b) I care more about the feelings of other people   

c) I am better at standing up for what I believe   

d) I tell the truth more often even when it is hard   

e) I am better at taking responsibility for my actions   

f) I am more interested in community and world problems   

g) I am better at speaking up for people who have been

treated unfairly   

ool

kit f

atin

itiv

th D

elo

[Sense of Self – For ASI grantees choosing this domain, all questions are required.]

11) Coming to [this program] has helped me to …

Yes Kind of Not Really

a) Feel better about myself   

b) Feel I have more control over things that happen to me   

c) Feel that I can make more of a difference   

d) Learn I can do things I didn’t think I could do before   

e) Feel better about my future   

f) Feel I am better at handling whatever comes my way   

[Optional]

12) What do you like best about coming to [this program]?

___________________________________________________________________________

[Optional]

13) If you could change one thing at [this program] what would it be?

___________________________________________________________________________

[Demographic questions 14-20 are required.]

14) How often do you come to [this

program]?

 Every day or almost everyday

(4-5 times per week)

 Two to three times a week

 Once a week

 A couple times a month

 Less than once a month

15) When did you start coming to [this

program]?

 Within the last 3 months

 4 to 6 months ago

 7 to 12 months ago

 More than 1 year ago

[FOR BASIC ONLY as an alternative to

#15.]

15a) When did you start coming to

[this program]?

 This fall

 Over the summer

 Last spring

 Last year or longer ago than that

16) How old are you?

 6 years or younger

 7 years

 8 years

 9 years

 10 years

 11 years

 12 years

 13 years

 14 years

 15 years or older

ool

kit f

atin

itiv

th D

elo

17) What grade are you in? _________

[Must be customized for each ASI grantee.]

20) Check each of the after-school

activities you participate in at [this

program].

18) Are you a girl/female or boy/male?

 Girl/ Female

 __________________________

 Boy/ Male

 __________________________

[Required FOR ADVANCED.]

 __________________________

19) Which best describes you?

 __________________________

 Caucasian/White

 African American/Black

 Asian/Pacific Islander

 Latino/Hispanic

 Native American

 Bi-racial/Multi-racial

 Other ______________

Thank you for your answers!

ool

kit f

atin

itiv

th D

elo

ENCUESTA DE JÓVENES PARTICIPANTES EN

[NAME OF ASI PROGRAM/AGENCY]

¡Danos tu opinión!

Esta encuesta se usa para obtener tus ideas sobre [este programa]. La encuesta no es una prueba ni

hay respuestas correctas ni incorrectas. Por favor, no tengas prisa en completarla y contesta cada

pregunta con tu opinión verdadera. Favor de NO poner tu nombre en la encuesta porque tus

respuestas son absolutamente privadas.

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

1) Favor de marcar el cuadrito con la descripción más cercana a tus sentimientos sobre

[este programa].

Sí Más o menos Casi nada/

Muy

poco

a) Hay actividades interesantes en [este programa]   

b) Las actividades son divertidas en [este programa]   

c) Aprendo muchas cosas nuevas en [este programa]   

d) Me gusta asistir a [este programa]   

e) Hay reglas que tengo que seguir en [este programa]   

f) Me siento seguro/a en [este programa]   

g) Creo que la gente aquí está feliz de verme   

h) Le digo a mis amigos que asistan a [este programa]   

2) Favor de marcar el cuadrito con la descripción más cercana a tus sentimientos sobre

[este programa].

Sí Más o menos Casi nada/

Muy

poco

a) Mi bienestar es importante para los empleados   

b) Los empleados me hacen sentir importante   

c) Los empleados me dan mucho apoyo   

d) Los empleados me dedican tiempo individualmente   

e) Tengo confianza en los empleados   

f) Me caen bien los empleados aquí   

g) Los empleados esperan que yo trate de ser lo más exitoso

posible   

h) Los empleados me dicen cuando hago algo correctamente   

i) Si tuviera un problema serio, yo me sentiría cómodo/a

pidiendo el consejo de un empleado de [este programa]   

j) Los empleados escuchan lo que digo   

k) Los empleados me piden planear, escoger o encabezar

actividades   

l) Los empleados me animan a trabajar de voluntario   

m) Los empleados tratan a todos los jóvenes de manera justa   

ool

kit f

atin

itiv

th D

elo

[Academic Success – For ASI grantees choosing this domain, required questions appear in

bold.]

3) El participar en [este programa] me ha ayudado a…

Sí Más o menos Casi nada/

Muy

poco

a) Mejorar mis calificaciones en la escuela   

b) Mejorar mi calificación de lenguaje (lectura/escritura)   

c) Mejorar mi calificación de matemáticas   

d) Mejorar mi calificación de ciencias   

e) Tener más éxito en mis estudios   

f) Tener más éxito en lenguaje (lectura/escritura)   

g) Tener más éxito en matemáticas   

h) Tener más éxito en ciencia   

i) Esforzarme más en mis estudios   

j) Dedicarle más tiempo a mi tarea   

k) Participar más en actividades escolares   

l) Aumentar mi interés en ir a la escuela   

m) Darle más importancia a mis estudios   

n) Llevarme mejor con mis compañeros de clase   

o) Llevarme mejor con mis maestros   

p) Pasar más tiempo leyendo por diversión, y no solamente

por obligación escolar   

[Arts & Recreation – For ASI grantees choosing this domain, question “sets” are required

separately: a-c, d-f or g-i.]

4) Por haber participado en [este programa]...

Sí Más o menos Casi nada/

Muy

poco

a) Dedico más tiempo a la artesanía   

b) Me gustan más las artesanías que antes   

c) Me desarrollo mejor en artesanía   

d) Ahora participo más en música, baile y/o teatro   

e) Ahora me gusta más el teatro, baile y/o la música   

f) Me desarrollo mejor en el teatro, baile y/o la música   

g) Ahora participo más en deportes, ejercicio y/o actividades

recreativas   

h) Ahora me gustan más los deportes, el ejercicio y/o actividades

recreativas   

i) Me desarrollo mejor en deportes, ejercicio y/o actividades

recreativas   

[Community Involvement – For ASI grantees choosing this domain, all questions are required.

FOR ADVANCED ONLY.]

5) El participar en [este programa] me ha ayudado a …

Sí Más o menos Casi nada/

Muy

poco

a) Pasar más tiempo como voluntario ayudando a gente de

mi comunidad   

b) Pasar más tiempo buscando la oportunidad de ayudar a

gente de mi comunidad   

c) Sentirme más significativo dentro de mi comunidad   

d) Sentir una conexión más grande con mi comunidad   

e) Sentirme mejor de mí mismo por ayudar a la gente   

ool

kit f

atin

itiv

th D

elo

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

ADVANCED ONLY.]

6) El involucrarme en [este programa] me ha ayudado a...

Sí Más o menos Casi nada/

Muy

poco

Tener más conocimiento de otras culturas, razas o grupos

étnicos   

b) Preocuparme más por el bienestar de jóvenes de otras

culturas, razas o grupos étnicos   

c) Tener más respeto para los jóvenes de otras culturas,

razas o grupos étnicos   

d) Sentir más confianza con los jóvenes de otras culturas,

razas o grupos étnicos   

e) Conversar más con los jóvenes quienes no hablan inglés

en su hogar   

f) Ser más consciente de no juzgar a personas por el color

de su piel   

g) Ser más consciente de que malinterpreto las ideas que

formo de otras personas y que no siempre son ciertas,

correctas o exactas   

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

ADVANCED ONLY.]

6a) Por haber participado en [este programa]...

Sí Más o menos Casi nada/

Muy

poco

a) Tengo más conocimiento de mi propia cultura, raza o

grupo étnico   

b) Me interesa más mi propia cultura, raza o grupo étnico   

c) Me siento más conectado con las tradiciones culturales

de mi familia   

d) Siento una conexión más grande con mi propia cultura,

raza o grupo étnico   

e) Siento más orgullo de mi propia cultura, raza o grupo

étnico   

f) Tengo más respeto hacia los miembros de mi propia

comunidad   

[Cultural Competency – For ASI grantees choosing this domain, all questions are required. FOR

BASIC ONLY.]

7) Por haber participado en [este programa]...

Sí Más o menos Casi nada/

Muy

poco

a) Tengo más conocimiento de las cosas buenas que se han

logrado por personas quienes tienen un semblante

diferente al mío. (Por ejemplo, Afro-Americanos,

México-Americanos, Nativo-Americanos, Asia-Americanos)   

b) Converso más con los jóvenes quienes tienen un

semblante diferente al mío   

c) Siento más confianza al estar con jóvenes quienes tienen

un semblante diferente al mío   

d) Comprendo que alguien quien tiene un semblante diferente

al mío, tal vez al fin de cuenta, no sea tan diferente   

ool

kit f

atin

itiv

th D

elo

[Life Skills – For ASI grantees choosing this domain, a, b, d, e and j are required FOR BASIC.

All questions are required FOR ADVANCED.]

8) Por haber participado en [este programa]...

Sí Más o menos Casi nada/

Muy

poco

a) Me comporto mejor con otras personas de mi misma edad   

b) Me desarrollo mejor al formar amistades   

c) Me desarrollo mejor al compartir mis ideas y sentimientos

con los demás   

d) Es más fácil entender a otras personas   

e) Es más fácil trabajar en equipo con otros   

f) Tomo mejores decisiones   

g) Me desarrollo mejor al planear por adelantado   

h) Me desarrollo mejor al establecer metas personales   

i) Me desarrollo mejor al solucionar problemas   

j) Me desarrollo mejor como líder   

k) Me desarrollo mejor al solucionar problemas sin violencia

ni enfrentamientos   

[Positive Life Choices – For ASI grantees choosing this domain, a, b and g are required. FOR

ADVANCED ONLY.]

9) El involucrarme en [este programa] me ha ayudado a...

Sí Más o menos Casi nada/

Muy

poco

a) Decir “no” a las actividades que son malas   

b) Evitar problemas   

c) Evitar el tabaco   

d) Evitar el alcohol   

e) Evitar las drogas   

f) Evitar las relaciones sexuales   

g) Evitar la violencia y pelear   

[Positive Core Values – For ASI grantees choosing this domain, a, b, c, and d are required FOR

BASIC. All questions are required FOR ADVANCED.]

10) Por haber participado en [este programa]...

Sí Más o menos Casi nada/

Muy

poco

a) Me importa más el bienestar de los demás   

b) Me preocupan más los sentimientos de otras personas   

c) Me desarrollo mejor al defender mis creencias   

d) Digo la verdad con más frecuencia, aunque sea difícil

hacerlo   

e) Puedo aceptar responsabilidad por mis propias acciones

con más facilidad   

f) Me interesan más los problemas de la comunidad

y del mundo   

g) Es más fácil defender a las personas que son tratadas

con injusticia   

ool

kit f

atin

itiv

th D

elo

[Sense of Self – For ASI grantees choosing this domain, all questions are required.]

11) El participar en [este programa] me ha ayudado a …

Sí Más o menos Casi nada/

Muy

poco

a) Sentirme mejor de mí mismo   

b) Sentirme más en control de lo que me pasa   

c) Sentir que puedo hacer una diferencia   

d) Entender que tengo habilidades que no conocía   

e) Sentirme mejor sobre mi futuro   

f) Tener más confianza para enfrentar el futuro   

[Optional]

12) ¿Qué es lo que más te gusta de [este programa]?

___________________________________________________________________________

[Optional]

13) ¿Cuál es una característica de [este programa] que te gustaría cambiar?

___________________________________________________________________________

[Demographic questions 14-20 are

required.]

14) ¿Cuándo asistes a [este programa]?

 Cada día o casi cada día

(de 4 a 5 veces por semana)

 De dos a tres veces por semana

 Una vez por semana

 Dos veces por mes

 Menos de una vez al mes

15) ¿Cuándo empezaste a asistir a [este

programa]?

 En los últimos 3 meses

 Hace 4 a 6 meses

 Hace 7 a 12 meses

 Hace más de un año

[FOR BASIC ONLY as an alternative to

#15.]

15a) ¿Cuándo empezaste a asistir a [este

programa]?

 Este otoño

 Durante el verano

 La primavera pasada

 El año pasado, o hace más de un

año

16) ¿Cuántos años tienes?

 6 años o menos

 7 años

 8 años

 9 años

 10 años

 11 años

 12 años

 13 años

 14 años

 15 años o más

ool

kit f

atin

itiv

th D

elo

17) ¿En qué grado estás? _________

[Must be customized for each grantee.]

20) Marque cada una de la actividad en la

cual participó durante el programa.

18) ¿Eres muchacha o muchacho?

 __________________________

 Muchacha

 __________________________

 Muchacho

 __________________________

[Required FOR ADVANCED.]

19) ¿Cuál es la mejor descripción de tu

grupo étnico?

 Caucásico/a / Blanco/a

 Africano americano/a / Negro/a

 Asiático/a / de las Islas del Pacífico

 Latino/a /Hispano/a

 Indio americano/a

 De dos razas / De más de dos razas

 Otra ______________

 __________________________

¡Gracias por tus respuestas!

ool

kit f

atin

itiv

th D

elo

Toolkit for Evaluating Positive Youth Development 92

Appendix X. Determining Which Survey to Use: Post-only versus Pre-post

Post-only survey question sets are recommended for use with the ASI programs for a number of reasons.

 Post-only survey administration creates less burden on respondents and staff.

 Measuring change using a pre-post format requires more response scale points to be sensitive enough to measure change, making

its use more difficult with participants of younger ages or lower literacy levels who are better served by 3-point scales.

 Post-only allows respondents to use their internal gauge to assess change rather than requiring that change be calculated by simple

math.

 Post-only is less susceptible to “ceiling” effects so that youth who consider themselves very strong at program start still can report

improvement in behavior or attitude (even if the change is subtle).

However, the post-only survey is considered to be weaker methodologically because it is more susceptible to bias and the promotion of

positive responses. When youth rate their status (on a behavioral or attitudinal outcome) at the end of a program, that self-report is

unlikely to be influenced by their memory of their rating several weeks earlier when the program began. The fact that youth are providing

unprompted responses (independent of earlier self-assessment) makes their final self-rating more credible. Consequently, when using the

pre-post design, be sure that youth are not reminded of their earlier ratings when they provide their post-program self-assessment.

Program leaders will need to decide which method works best for their program. There are a number of factors to consider. Some are

listed in the table below.

Factors to Consider Questions to Consider

Age and literacy level

of participants

Are your participants able to answer the questions on the pre-post instrument in a trustworthy way? (The pre-post

version is only recommended for participants in grade 5 and above.)

Program resources

Do you have the staff resources to administer the survey twice? It may be particularly difficult if you frequently add

new participants and must track each one for a unique pre- and post-period rather than enrolling groups at the start

of each semester.

Outcomes of interest

Are many of your outcomes those that may be susceptible to a ceiling effect? (Outcomes for which participants

report high levels at baseline will show less change in a pre-post format; e.g. honesty, caring.)

Need for more or less

rigorous outcome data

Do you want stronger evidence? Do you need stronger evidence for your funders? Are you skeptical of self-

reported post-only data?

Stage of program

development

Has your program been around long-enough that you are ready for a more rigorous outcome evaluation? (New and

developing programs might benefit more from the post-only format until they have stronger roots at which time

they may move to pre-post format.)

Flowchart for Selecting Post-only or Pre-post Evaluation Design

The flow chart below is designed to guide staff in determining whether a post-only or a pre-post method would work best for their after-

school program and participants. It provides one framework for determining evaluation design, however, programs may choose to adjust

the order to fit their needs.

New Established

Post-

Only

Design

< 5th Grade 5th Grade +

Low Adequate

Ceiling Effect on Outcomes

Assessed

Likely No

No Yes

Pre-Post

Design

Post-

Only

Design

Post-

Only

Design

Post-

Only

Design

Post-

Only

Design

Program Resources

Developmental Stage of

Program

Youth Literacy Level

Audience Requires

Rigorous Assessment

ool

kit f

atin

itiv

th D

elo

Toolkit for Evaluating Positive Youth Development 94

Comparison of Question Sets: Pre-Post to Post-Only Categorized by Outcome

Outcome Domain Post-Only Question Pre-Post Question Scale

Post-Only Scale Legend

Yes, Kind Of, Not Really

Pre-Post Scale Legend

Frequency: All of the time, most of the time, some

of the time, never, don’t know

Hours: 0 hours, 1 hour, 2 hours, 3 hours, 4 hours, 5

or more hours, don’t know

Amount: A lot, some, very little, none, don’t know

Grades: Mostly A's , mostly B's, mostly C's, mostly

D's, mostly F's, none of these grades, not sure

Academic Success

Coming to [this program] has helped me to…

Improve my grades in school. During the past semester, how would you describe

your grades in school?

Grades

Improve my grade in language arts (reading/writing). During the past semester, how would you describe

your grades in language arts (reading/writing)?

Grades

Improve my grade in math. During the past semester, how would you describe

your grades in math?

Grades

Improve my grade in science. During the past semester, how would you describe

your grades in science?

Grades

Do better in school. I do well in school. Frequency

Do better in language arts (reading/writing). I do well in language arts (reading/writing). Frequency

Do better in math. I do well in math. Frequency

Do better in science. I do well in science. Frequency

Try harder in school. I try hard in school. Frequency

Spend more time doing my homework. During the past 7 days, about how many hours did

you spend doing homework outside of school?

Hours

Participate more in class activities. I participate in class activities. Frequency

Become more interested in going to school. I like school or I am interested in going to school. Frequency

Care more about my school. I care about my school. Frequency

Get along better with my classmates. I get along with my classmates. Frequency

Get along better with my teachers. I get along with my teachers. Frequency

Spend more time reading for fun (not school). During the past 7 days, about how many hours do

you spend reading for fun (not for school)?

Hours

Outcome Domain Post-Only Question Pre-Post Question Scale

Arts and Recreation

Because I came to [this program] …

I spend more time doing arts and crafts. During the past 7 days, about how many hours did

you spend in arts and crafts?

Hours

I enjoy arts and crafts more. I enjoy arts and crafts. Frequency

I am better in arts and crafts. I am good at arts and crafts. Frequency

I spend more time in music, dance and/or theater. During the past 7 days, about how many hours did

you spend in music, dance and/or theater?

Hours

I enjoy music, dance and/or theater more. I enjoy music, dance and/or theater. Frequency

I am better at music, dance and/or theater. I am good at music, dance and/or theater. Frequency

I spend more time doing sports, exercise and/or

recreation activities.

During the past 7 days, about how many hours did

you spend in sports, exercise and/or recreation

activities?

Hours

I enjoy sports, exercise and/or recreation activities

more.

I enjoy sports, exercise and/or recreation activities. Frequency

I am better at sports and/or recreation activities. I am good at sports, exercise and/or recreation

activities.

Frequency

Community

Involvement

Coming to [this program] has helped me…

Spend more time volunteering or helping others in

my community.

During the past 7 days, about how many hours did

you spend volunteering or helping others in your

community?

Hours

Spend more time looking for opportunities to help

others in my community.

During the past 7 days, how many hours did you

spend looking for opportunities to help others in

your community?

Hours

Feel more important to my community. I am important to my community. Frequency

Feel a stronger connection to my community. I feel a strong connection to my community. Frequency

Feel better about myself because I help others. I feel good about myself because I help others. Frequency

Toolkit for Evaluating Positive Youth Development 95

Toolkit for Evaluating Positive Youth Development 96

Outcome Domain Post-Only Question Pre-Post Question Scale

Cultural

Competency

Being involved in [this program] has helped me

to…

I know more about the good things that people who

look or sound different than me have done (like

African Americans, Mexican Americans, Native

Americans, Asian Americans and others)

N/A Frequency

I talk more with young people who look or sound

different than me.

N/A

I feel more comfortable being around young people

who look or sound different than me.

N/A Frequency

(Basic set)

I understand that someone who looks or sounds

different than me may not be that different after all

N/A Frequency

Being involved in [this program] has helped me

to …

Know more about people of other cultures, races or

ethnic groups.

How much do you think you know about people of

other cultures, races or ethnic groups?

Amount

Care more about young people of other cultures,

races or ethnic groups.

I care about young people of other cultures, races or

ethnic groups.

Frequency

Have more respect for young people of other

cultures, races or ethnic groups.

I have respect for young people of other cultures,

races or ethnic groups.

Frequency

Feel more comfortable with young people of other

cultures, races or ethnic groups.

I feel comfortable with young people of other

cultures, races or ethnic groups.

Frequency

Talk more to young people who speak languages

other than English at home.

During the past 7 days, about how many hours did

you spend talking to young people who speak

languages other than English at home?

Hours

Try harder not to judge people based on skin color. I try not to judge people based on skin color. Frequency

(Advanced set)

Be more aware that I sometimes have ideas about

people that are not true (Incorrect or mistaken)

I realize that I have ideas about people that are not

true (incorrect or mistaken).

Frequency

Outcome Domain Post-Only Question Pre-Post Question Scale

Because I came to [this program]…

I know more about my own culture, race or ethnic

group.

How much do you know about your own culture,

race or ethnic group?

Amount

I am more interested in my own culture, race or

ethnic group.

I am interested in my own culture, race or ethnic

group.

Frequency

I feel more connected to my family’s cultural

traditions.

I feel connected to my family’s cultural traditions. Frequency

I feel more connected to my own culture, race or

ethnic group.

I feel connected to my own culture, race or ethnic

group.

Frequency

I feel more pride in my own culture, race or ethnic

group.

I feel pride for my own culture, race or ethnic

group.

Frequency

(Advanced set)

I have more respect for members of my own

community

I respect members of my own cultural, racial or

ethnic group.

Frequency

Life Skills

Because I came to [this program]…

I get along better with other people my age. I get along with other people my age. Frequency

I am better at making friends. I am good at making friends. Frequency

I am better at telling others about my ideas and

feelings.

I am good at telling others about my ideas and

feelings.

Frequency

I am better at listening to other people. I am good at listening to other people. Frequency

I work better with others on a team. I work well with others on a team. Frequency

I make better decisions. I make good decisions. Frequency

I am better at planning ahead. I am good at planning ahead. Frequency

I am better at setting goals. I am good at setting goals. Frequency

I am better at solving problems. I am good at solving problems. Frequency

I am more of a leader. I am a good leader. Frequency

I am better at taking care of problems without

violence or fighting.

I am good at taking care of problems without

violence or fighting.

Frequency

Positive Core

Values

Because I came to [this program]…

I care more about other people. I care about other people. Frequency

I care more about the feelings of other people. I care about the feelings of other people. Frequency

Toolkit for Evaluating Positive Youth Development 97

Toolkit for Evaluating Positive Youth Development 98

Outcome Domain Post-Only Question Pre-Post Question Scale

I am better at standing up for what I believe. I stand up for what I believe. Frequency

I tell the truth more often even when it is hard. I tell the truth more often even when it is hard. Frequency

I am better at taking responsibility for my actions. I take responsibility for my actions. Frequency

I am more interested in community and world

problems.

I am interested in community and world problems. Frequency

I am better at speaking up for people who have been

treated unfairly.

I speak up for people who have been treated

unfairly.

Frequency

Positive Life

Choices

Coming to [this program] has helped me to…

Better say “no” to things I know are wrong. I say “no” to things I know are wrong. Frequency

Stay out of trouble. I stay out of trouble. Frequency

Stay away from tobacco. I stay away from tobacco. Frequency

Stay away from alcohol. I stay away from alcohol. Frequency

Stay away from drugs. I stay away from drugs. Frequency

Stay away from sex. I stay away from sex. Frequency

Stay away from violence and fighting. I stay away from violence and fighting. Frequency

Sense of Self

Coming to [this program] has helped me to…

Feel better about myself. I feel good about myself. Frequency

Feel I have more control over things that happen to

me.

I feel I have control over things that happen to me. Frequency

Feel that I can make more of a difference. I feel that I can make a difference. Frequency

Learn I can do things I didn’t think I could do

before.

I am good at learning new things. Frequency

Feel better about my future. I feel good about my future. Frequency

Feel I am better at handling whatever comes my way. I am good at handling whatever comes my way. Frequency

Appendix XI. Complete Pre-Post Toolkit Survey Question Set

The pre-post survey question sets ask youth participants to rate themselves on their current status. This rating is made twice – once at

program start (pre-program) and once at program end (post-program). This method of questioning varies from the post-only question sets

wherein youth participants are asked at program end to rate the amount of change they have made from program start to program end. An

example question asked in each format is presented in the following table:

Pre-Post Version Post-Only Version

(Administered at onset of program)

Pre: I feel a strong connection to my community

 All of the time

 Most of the time

 Some of the time

 Never

(Administered at end of program)

Post: I feel a strong connection to my community

 All of the time

 Most of the time

 Some of the time

 Never

Analysis: Post-test rating is compared to pre-test rating to see if

any movement occurred.

(Administered at end of program)

Post: Coming to this program has helped me to feel a stronger

connection to my community

 Yes

 Kind of

 Not Really

The pre-post version is generally considered a stronger method for evaluating youth participants; however, it has a couple of limitations.

The pre-post versions of the toolkit question sets are appropriate for youth participants with higher levels of literacy because it uses more

expansive question scales (shifting from 3-point scale to 4- and 5-point scales). In addition, special care is needed when planning the

administration because the pre-test surveys must be administered as close to program start as possible in order to be as accurate as possible

(i.e., the participant may have experienced change within the first few weeks of the program). Appendix X provides an in-depth

comparison of the pre-post survey to the post-only survey.

The pre-post questions are only available for measuring outcomes. Program satisfaction can only be collected in the post-only format.

Reliability testing was not conducted on the pre-post survey question set due to the limited sample size available at the time of publication.

Toolkit for Evaluating Positive Youth Development 99

[NAME OF ASI PROGRAM/AGENCY]

YOUTH PARTICIPANT SURVEY

Tell us what you think!

This survey is being used to get your ideas about [this program]. It is not a test and there are no right

or wrong answers. Please take your time and be sure to answer each question based on what you

really think. Please do NOT put your name on this survey. Your answers are completely private.

[ONLY USE THE PROGRAM QUALITY QUESTION 1a ON POST SURVEY.]

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

1a) Please check the box below that is closest to how you feel about [this program].

All of Most of Some of Don’t

the

time the time the time Never know

a) There are interesting activities at [this program].     

b) The activities are fun at [this program].     

c) I learn about new things at [this program].     

d) I like coming to [this program].     

e) There are rules I am expected to follow at [this

program].     

f) I feel safe at [this program].     

g) I feel like people are happy to see me here.     

h) I tell my friends to come to [this program].     

[ONLY USE PROGRAM QUALITY QUESTION 2a ON POST SURVEY.]

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

2a) Please check the box below that is closest to how you feel about [this program].

All of Most of Some of Don’t

the

time the time the time Never know

a) Staff care about me.     

b) Staff help me feel important.     

c) Staff give me lots of support.     

d) Staff spend time with me one-on-one.     

e) Staff can be trusted.     

f) I like the staff here.     

g) Staff expect me to try hard and do my best.     

h) Staff tell me when I do a good job.     

i) I could go to a staff member at [this program] for

advice if I had a serious problem.     

j) Staff listen to what I have to say.     

k) Staff ask me to plan, choose or lead activities.     

l) Staff encourage me to do volunteer work.     

m) Staff treat all kids fairly.     

Toolkit for Evaluating Positive Youth Development 100

[Academic Success – For ASI grantees choosing this domain, required questions appear in

bold.]

1) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the time the time the time Never know

a) I do well in school.     

b) I do well in language arts (reading/writing).     

c) I do well in math.     

d) I do well in science.     

e) I try hard in school.     

f) I participate in class activities.     

g) I like school or I am interested in going to school.     

h) I care about my school.     

i) I get along with my classmates.     

j) I get along with my teachers.     

[Arts & Recreation – For ASI grantees choosing this domain, question “sets” are required

separately: a-b, c-d or e-f.]

2) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I enjoy arts and crafts.

    

b) I am good at arts and crafts.     

c) I enjoy music, dance and/or theater.

    

d) I am good at music, dance and/or theater.     

e) I enjoy sports, exercise and/or recreation activities.

    

f) I am good at sports, exercise and/or recreation

activities.     

[Community Involvement – For ASI grantees choosing this domain, all questions are required.]

3) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I am important to my community.     

b) I feel a strong connection to my community.     

c) I feel good about myself because I help others.     

[Cultural Competency – For ASI grantees choosing this domain, all questions are required.]

4) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I care about young people of other cultures, races

or ethnic groups.

    

b) I have respect for young people of other cultures,

races or ethnic groups.     

c) I feel comfortable with young people of other

cultures, races or ethnic groups.

    

d) I try not to judge people based on skin color.     

e) I realize that I have ideas about people that are not

true (incorrect or mistaken).

    

Toolkit for Evaluating Positive Youth Development 101

[Cultural Competency – For ASI grantees choosing this domain, all questions are required.]

4a) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I am interested in my own culture, race or ethnic

group.

    

b) I feel connected to my family’s cultural traditions.     

c) I feel connected to my own culture, race or ethnic

group.

    

d) I feel pride in my own culture, race or ethnic group.     

e) I respect members of my own cultural, racial or

ethnic community.

    

[Life Skills – For ASI grantees choosing this domain, a, b, d, e and j are required FOR BASIC.

All questions are required FOR ADVANCED.]

5) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I get along with other people my age.

    

b) I am good at making friends.     

c) I am good at telling others about my ideas and

feelings

.     

d) I am good at listening to other people.     

e) I work well with others on a team

.     

f) I make good decisions.     

g) I am good at planning ahead.     

h) I am good at setting goals.     

i) I am good at solving problems

.     

j) I am a good leader.     

k) I am good at taking care of problems without

violence or fighting

.     

[Positive Life Choices – For ASI grantees choosing this domain, a, b and g are required.]

6) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the time the time the time Never know

a) I say “no” to things I know are wrong.     

b) I stay out of trouble.     

c) I stay away from tobacco.     

d) I stay away from alcohol.     

e) I stay away from drugs.     

f) I stay away from sex.     

g) I stay away from violence and fighting.     

Toolkit for Evaluating Positive Youth Development 102

[Positive Core Values - All questions are required.]

7) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I care about other people.     

b) I care about the feelings of other people.     

c) I stand up for what I believe.     

d) I tell the truth more often even when it is hard.     

e) I take responsibility for my actions.     

f) I am interested in community and world problems.     

g) I speak up for people who have been treated unfairly.     

[Sense of Self – All questions are required.]

8) Please check the box below that is closest to how you feel…

All of Most of Some of Don’t

the

time the time the time Never know

a) I feel good about myself.     

b) I feel I have control over things that happen to me.     

c) I feel that I can make a difference.     

d) I am good at learning new things.     

e) I feel good about my future.     

f) I am good at handling whatever comes my way.     

[Questions a and b are required if choosing the Academic Success domain; f and g are required

if choosing the Community Involvement domain; h is required if choosing the Cultural

Competency domain.]

9) During the past 7 days, about how many hours did you spend…

0 1 2 3 4 5 Don’t

hours hour hours hours hours hours know

a) Doing homework outside of school?       

b) Reading for fun (not for school)?       

c) In arts and crafts?       

d) In music, dance and/or theater?       

e) In sports, exercise and/or recreation activities?       

f) Volunteering or helping others in your community?       

g) Looking for opportunities to help others in your

community?       

h) Talking to young people who speak languages other

than English at home?       

[Question a is required if choosing the Academic Success domain.]

10) During the past semester, how would you describe your grades…

Mostly Mostly Mostly Mostly Mostly None of Not

A’s B’s C’s D’s F’s these grades sure

a) In school?       

b) In language arts (reading/writing)?       

c) In math?       

d) In science?       

Toolkit for Evaluating Positive Youth Development 103

[Questions a and b are required if choosing the Cultural Competency domain.]

11) How much do you…

A Very Don’t

lot

Some little None know

a) Think you know about people of other cultures, races

or ethnic groups?     

b) Know about your own culture, race or ethnic group?     

[Questions 12 and 13 are omitted on the pre-program survey, and OPTIONAL on the post-only

survey.]

[Demographic questions 14-20 are required.]

14) How often do you come to [this

program]?

 Every day or almost everyday

(4-5 times per week)

 Two to three times a week

 Once a week

 A couple times a month

 Less than once a month

15) When did you start coming to [this

program]?

 Within the last 3 months

 4 to 6 months ago

 7 to 12 months ago

 More than 1 year ago

16) How old are you?

 6 years or younger

 7 years

 8 years

 9 years

 10 years

 11 years

 12 years

 13 years

 14 years

 15 years or older

17) What grade are you in? _________

18) Are you a girl/female or boy/male?

 Girl/Female

 Boy/Male

19) Which best describes you?

 Caucasian/White

 African American/Black

 Asian/Pacific Islander

 Latino/Hispanic

 Native American

 Bi-racial/Multi-racial

 Other ______________

[Must be customized for each ASI grantee.]

20) Check each of the after-school

activities you participate in at [this

program].

 __________________________

Thank you for your answers!

Toolkit for Evaluating Positive Youth Development 104

[NAME OF ASI PROGRAM/AGENCY]

ENCUESTA PARA JOVENES PARTICIPANTES

¡Danos tu opinión!

Esta encuesta se usa para obtener tus ideas sobre [este programa]. La encuesta no es una prueba ni

hay respuestas correctas ni incorrectas. Por favor, no tengas prisa en completarla y contesta cada

pregunta con tu opinión verdadera. avor de NO poner tu nombre en la encuesta porque tus respuestas

son absolutamente

rivadas.

[ONLY USE PROGRAM QUALITY QUESTION 1a ON POST SURVEY.]

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

1a) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Hay actividades interesantes en [este programa].     

b) Las actividades son divertidas en [este programa].     

c) Aprendo muchas cosas nuevas en [este programa].     

d) Me gusta asistir a [este programa].     

e) Hay reglas que tengo que seguir en [este programa].     

f) Me siento seguro/a en [este programa].     

g) Creo que la gente aquí está feliz de verme.     

h) Le digo a mis amigos que asistan a [este programa].     

[ONLY USE PROGRAM QUALITY QUESTION 2a ON POST SURVEY.]

[Program Quality – Required questions appear in bold and must be included by all ASI

grantees.]

2a) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Mi bienestar es importante para los empleados.     

b) Los empleados me hacen sentir importante.     

c) Los empleados me dan mucho apoyo.     

d) Los empleados me dedican tiempo individualmente.     

e) Tengo confianza en los empleados.     

f) Me caen bien los empleados aquí.     

g) Los empleados esperan que yo trate de ser lo más

exitoso posible.     

h) Los empleados me dicen cuando hago algo

correctamente.     

i) Si tuviera un problema serio, yo me sentiría cómodo/a

pidiendo el consejo de un empleado de [este

programa].     

j) Los empleados escuchan lo que digo.     

k) Los empleados me piden planear, escoger o encabezar

actividades.     

l) Los empleados me animan a trabajar de voluntario.     

m) Los empleados tratan a todos los jóvenes de manera

justa.     

Toolkit for Evaluating Positive Youth Development 105

[Academic Success – For ASI grantees choosing this domain, required questions appear in

bold.]

1) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Tengo buenos resultados en la escuela.     

b) Tengo buenos resultados en lenguaje (lectura/escritura).     

c) Tengo buenos resultados en matemáticas.     

d) Tengo buenos resultados en ciencia.     

e) Hago lo mejor que puedo en la escuela.     

f) Participo en actividades de la clase.     

g) Me gustan los estudios o tengo interés en ir a la

escuela.     

h) Me importan mis estudios.     

i) Tengo buenas relaciones con mis compañeros de la

escuela.     

j) Tengo buenas relaciones con mis maestros.     

[Arts & Recreation – For ASI grantees choosing this domain, question “sets” are required

separately: a-b, c-d or e-f.]

2) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Me gustan las artesanías y manualidades.     

b) Soy bueno para las artesanías y manualidades.     

c) Me gusta la música, el baile y/o el teatro.     

d) Soy bueno para la música, el baile y/o el teatro.     

e) Me gustan los deportes, el ejercicio y/o las actividades

recreativas.     

f) Soy bueno para los deportes, el ejercicio y/o las

actividades recreativas.     

[Community Involvement – For ASI grantees choosing this domain, all questions are required.]

3) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Soy importante para mi comunidad.     

b) Me siento muy conectado con mi propia comunidad.     

c) Me siento bien conmigo mismo porque ayudo a los

demás.     

Toolkit for Evaluating Positive Youth Development 106

[Cultural Competency – For ASI grantees choosing this domain, all questions are required.]

4) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre veces veces Nunca sé

a) Me interesa la juventud de otras culturas, razas y

grupos étnicos.     

b) Tengo respeto hacia la juventud de otras culturas,

razas o grupos étnicos.     

c) Me siento cómodo con la juventud de otras culturas,

razas o grupos étnicos.     

d) Trato de no juzgar a la gente por su color de piel.     

e) Estoy consiente de que tengo perspectivas de otra

gente que no son verdaderas (incorrectas o

equivocadas).     

[Cultural Competency – For ASI grantees choosing this domain, all questions are required.]

4a) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre veces veces Nunca sé

a) Me interesa mi propia cultura, raza o grupo étnico.     

b) Siento una conexión con las tradiciones culturales

de mi familia.     

c) Siento una conexión con mi propia cultura, raza o

grupo étnico.     

d) Siento orgullo de mi propia cultura, raza o grupo étnico.    

e) Respeto a los miembros de de mi propia comunidad

cultural, racial y étnica.     

[Life Skills – For ASI grantees choosing this domain, a, b, d, e and j are required FOR BASIC.

All questions are required FOR ADVANCED.]

5) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Tengo buenas relaciones con otras personas de mi

edad.     

b) Para mi es fácil tener amistades.     

c) Para mi es fácil compartir mis ideas y sentimientos.     

d) Para mi es fácil escuchar a otras personas.     

e) Cuando estamos en grupo, trabajo bien con las otras

personas de mi equipo.     

f) Tomo buenas decisiones.     

g) Soy bueno al planear para el futuro.     

h) Soy bueno para establecer metas.     

i) Soy bueno para resolver problemas.     

j) Soy un buen líder.     

k) Soy bueno para resolver problemas sin usar violencia

o sin tener que pelear.     

Toolkit for Evaluating Positive Youth Development 107

[Positive Life Choices – For ASI grantees choosing this domain, a, b and g are required.]

6) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre veces veces Nunca sé

a) Cuando yo sé que algo es malo, digo que “no”.     

b) Me mantengo libre de problemas.     

c) Me mantengo libre de tabaco.     

d) Me mantengo libre de alcohol.     

e) Me mantengo libre de drogas.     

f) Me mantengo libre de sexo.     

g) Me mantengo libre de la violencia y de peleas.     

[Positive Core Values - All questions are required.]

7) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Me importa el bienestar de otras personas.     

b) Me importan los sentimientos de otras personas.     

c) Cuando creo en algo, defiendo mi punto de vista.     

d) Aunque sea difícil, digo la verdad.     

e) Tomo responsabilidad por mis acciones.     

f) Me importan los problemas de la comunidad y del

mundo.     

g) Defiendo a las personas cuando son maltratadas.     

[Sense of Self – All questions are required.]

8) Por favor marque la respuesta que más corresponda con su opinión personal sobre

[este programa].

Muchas Algunas No

Siempre

veces veces Nunca sé

a) Me siento bien de mi mismo.     

b) Siento que tengo control sobre las cosas que me

suceden.     

c) Siento que puedo hacer una diferencia.     

d) Soy bueno para aprender cosas nuevas.     

e) Me siento bien de mi futuro.     

f) Soy bueno para enfrentar todo lo que me espera en mi

vida.     

Toolkit for Evaluating Positive Youth Development 108

[Questions a and b are required if choosing the Academic Success domain; f and g are required

if choosing the Community Involvement domain; h is required if choosing the Cultural

Competency domain.]

9) Durante los últimos 7 días, ¿aproximadamente cuantas horas dedicaste a sobre [este

programa].

0 1 2 3 4 5 No

horas hora horas horas horas horas sé

a) la tarea en un lugar que no haya sido la escuela?       

b) la lectura como diversión (no como tarea escolar)?       

c) las artesanías y manualidades?       

d) la música, el baile y/o el teatro?       

e) los deportes, el ejercicio y/o las actividades

recreativas?       

f) darte de voluntario o en ayudar a otros de tu

comunidad?

g) buscar oportunidades para ayudar a otros de tu

comunidad?       

h) conversar con jóvenes que no hablan inglés en sus

hogares?       

[Question a is required if choosing the Academic Success domain.]

10) Durante el último semestre, ¿qué tipo de calificaciones, notas o grados recibiste

sobre [este programa].

La La La La La Ninguna No

mayoría mayoríamayoríamayoría mayoría de estas estoy

A B C D F

notas seguro

a) en la escuela?       

b) en lenguaje (lectura/escritura)?       

c) en matemáticas?       

d) en ciencia?       

[Questions a and b are required if choosing the Cultural Competency domain.]

11) ¿Cuánto…

Muy No

Mucho Poco poco Nada sé

a) piensas saber acerca de personas de otras culturas,

razas o grupos étnicos?     

b) sabes acerca de tu propia cultura, raza o grupo étnico?     

[Questions 12 and 13 are omitted on the pre-program survey, and OPTIONAL on the post-only

survey.]

[Demographic questions 14-20 are required.]

14) ¿Cuándo asistes a [este programa]?

 Cada día o casi cada día

(de 4 a 5 veces por semana)

Toolkit for Evaluating Positive Youth Development 109

 De dos a tres veces por semana

 Una vez por semana

 Dos veces por mes

 Menos de una vez al mes

15) ¿Cuándo empezaste a asistir a [este

programa]?

 En los últimos 3 meses

 Hace 4 a 6 meses

 Hace 7 a 12 meses

 Hace más de un año

19) ¿Cuál es la mejor descripción de tu

grupo étnico?

16) ¿Cuántos años tienes?

 6 años o menos

 7 años  Caucásico/a / Blanco/a

 8 años  Africano americano/a / Negro/a

 9 años  Asiático/a / de las Islas del Pacífico

 10 años  Latino/a /Hispano/a

 1 años  Indio americano/a

 12 años  De dos razas / De más de dos razas

 13 años  Otra ______________

 14 años

 15 años o más

[Must be customized for each ASI grantee.]

20) Marque cada una de la actividad en la

cual participó durante [este programa].

17) ¿En qué grado estás? _________

 __________________________

18) ¿Eres muchacha o muchacho?

 Muchacha  __________________________

 Muchacho  __________________________

 __________________________

¡Gracias por tus respuestas!

Toolkit for Evaluating Positive Youth Development 110

Toolkit for Evaluating Positive Youth Development 111

REFERENCES

The International Council of Self-Esteem. Available at: http://www.self-esteem-international.org. Accessed April 1, 2004.

Strein W. Assessment of Self-Concept. ERIC Digest [serial online]. 1995;071. Available at:

http://www.ericfacility.net/ericdigests/ed389962.html. Accessed April 1, 2004.

Tucker-Ladd C. Psychological Self-Help. Mental Health Net [serial online]. Available at: http://www.mentalhelp.net/psyhelp/. Accessed

April 1, 2004.

Quinn J. Where Need Meets Opportunity: Youth Development Programs for Early Teens. The Future of Children: When School is Out. Fall

1999;9:96-116

Leffert N, Saito RN, Blyth DA, Kroenke CH. Making the case: Measuring the impact of youth development programs. Minneapolis, MN: Search

Institute; 1996.

Judd B. Elements of Successful Youth Programs. Alaska Division of Public Health; 2000.

Eisen M, Pallitto C, Bradner C, Bolshun N. Teen Risk-Taking: Promising Prevention Programs and Approaches. Washington, D.C.: Urban

Institute; 2000.

Leffert N, Benson PL, Scales, PC, Sharma AR, Drale DR, Blyth DA. Developmental assets: Measurement and prediction of risk behaviors among

adolescents (Draft). Minneapolis, MN: Search Institute; 1998.

Pittman KJ, Cahill M. A new vision: Promoting youth development. Paper commissioned by the Center for Youth Development and Policy

Research, Academy for Educational Development. Washington, D.C.; 1991.

Search Institute 40 Developmental Assets. Available at: http://www.search-institute.org/assets/. Accessed April 1, 2004.

Litwin, MS. How to measure survey reliability and validity. Thousand Oaks, CA: Sage Publications, Inc; 1995.

Robinson JP, Shaver PR, Wrightsman LS. Measures of personality and social psychological attitudes. San Diego, CA: Academic Press, Inc; 1991.

THE COLORADO TRUST

1600 SHERMAN STREET | DENVER, CO 80203-1604 | 303-837-1200 | 888-847-9140 | WWW.COLORADOTRUST.ORG