Boys and Girls Clubs



Home > College of Education > Secondary Education > SEDEBA > Youth Substance Abuse Prevention > Boys and Girls Clubs

Manatee County Children's Services

Youth Substance Abuse Prevention
Program Evaluation

Program Plan Report

Agency: Boys and Girls Clubs of Manatee County

Program: Smart Moves

PROFILE

Comprehensive Programming /Program Process

Needs Assessments

Outcomes Design

Prevention Program Theory

DIAGNOSTIC

Needs Assessment/Outcomes Design Match

Outcomes Design/Outcome Data Match

Prevention Program Theory/Evidence-Based Programs Match

Calculations of Cost-Effectiveness

ACTION PLAN

PROFILE

Comprehensive Programming/Program Process

Target Population: All participants in the Club, either through the year or during the summer. Enroll by enrolling in the Club.

Number of Participants Served: 400+ now (ranges from 250-400 through the year).

Main Goal(s): The main goal is to provide a safe and positive environment with SMART Moves as a facet of the overall program

Activities Conducted or Services Provided: Classes.

Qualifications/Credentials of Prevention Specialists: Florida Department of Children and Families (DCF) license. Curriculum does not provide training.

General Approach: Classroom approach. Goal sheets in each kid’s folder. Supplemented with worksheets, online ideas. Coloring – samples put on board.

Schedule of Activities: There is a progression of activities in the curriculum. (10 weeks.) Students grouped by age. Each student usually gets Smart Moves twice a week.

Reinforcement: By repetition from one year to the next.

Academic Support: Homework time (Project Learn). Teacher-taught and peer learning. Targeted outreach to high-risk kids (as identified from information from parents or schools).

Substances Addressed: Alcohol, tobacco, marijuana, prescription drugs. Also, HIV/AIDS, sex.

Constructive/Healthy Alternatives to Substance Abuse Offered or Promoted: Alternative activities include Project Learn, environmental/earth science, basketball, flag football, soccer, arts and crafts. The Team Room has a sound room to record CDs.

Opportunities for Youth Participation in the Community: Community cleanups.

Knowledge about substance abuse addressed? Yes. Knowledge and attitudes about the dangers, and skills in handling peer pressure. For the younger kids (6-7 year olds), just making them aware it exists.

Attitudes about substance abuse addressed? Yes. (see above)

Substance abuse behavior addressed? No.

Refusal skills/peer resistance taught? Yes.

Negative media messages addressed? Yes.

Stress management/anger control/self-control addressed? Yes.

Decision-making skills/critical thinking addressed? Yes.

Self-esteem addressed? Yes. Work with self esteem with older kids.

High and positive life goals promoted? Yes. Start with life goals even with young kids – long-term consequences of decisions with 2nd graders.

Self-efficacy/sense of personal power/personal optimism promoted? Yes.

Other issues addressed?

Opportunity for youth leadership with peers or in community?
Yes. Leaders in Training at two clubs with other teen leadership through environmental programs.

Recognize program participants for achievement? (e.g., awards, celebrations)

Yes. Banquets for sports events after the season ends. Monthly Youth of the Month Recognition Dinner. Annual Youth of the Year competition.

Needs Assessments

Needs Assessments Done: Intake forms, pretests.

Evidence Target Population is At-Risk: Nextdoor is a drug house. Poverty in neighborhood, Parents enroll kids at local elementary school.

Highest Priority Risk Factors (from Online Tools):

(Children 9+) Alcohol, tobacco, smokeless tobacco, and to a lesser extent, marijuana/hashish, Ecstasy, and prescription pain-relievers, including Oxycontin. Perception of risk in alcohol, regular smoking of cigarettes, and smoking marijuana.

(Children under 9) Problem-solving skills, defending views under group pressure, and education and learning problems.

Outcomes Design

Outcomes from Online POM Database: Increased Knowledge of HIV, AIDS and negative consequences of substance abuse and pre-mature sexual activity.

Targets from Online POM Database: 90% of the participants will increase their knowledge about the negative consequences of alcohol, drugs and early sexual activity, as measured by pre and post-tests given with the approximately 10-week course

Prevention Program Theory

Replicated Program? Yes.

Theory: From the website (http://www.bgca.org/programs/healthlife.asp) : “The SMART Moves (Skills Mastery and Resistance Training) prevention/education program addresses problems such as drug and alcohol use and premature sexual activity. The program uses a team approach involving Club staff, peer leaders, parents and community representatives. More than simply emphasizing a "Say No" message, the program teaches young people ages 6 to15 how to say no by involving them in discussion and role-playing, practicing resistance and refusal skills, developing assertiveness, strengthening decision-making skills and analyzing media and peer influence. The ultimate goal: to promote abstinence from substance abuse and adolescent sexual involvement through the practice of responsible behavior.”

Evidence Basis

Assessments or Data Sources: Pretest/posttest scores from curriculum pretest/posttest.

DIAGNOSTIC

Needs Assessment/Outcomes Design Match

The risk factor of perception of risk in abusing alcohol, marijuana, and cigarettes is matched by the outcome of increasing knowledge about the negative consequences of substance abuse.

Perception however is subtly different from knowledge. A stronger match would be an outcome related to affecting attitudes and perceptions about the danger of substance abuse. Apparently, the program addresses attitudes although it does not make this an outcome.

Also, the risk factors of behavior in children 9+ are not matched with any outcomes, nor are the risk factors of children under 9 -- problem-solving skills, defending views under group pressure, and education and learning problems – matched with outcomes, even though the program apparently cultivates decision-making skills, critical thinking skills, and refusal skills, and provides academic support.

Knowledge of substance abuse is not a risk factor for children under 9 even though the program addresses this.

Outcomes Design/Outcome Data Match

Pretest/posttest scores from curriculum pretest/posttest. Do these measure only knowledge? The outcome using these scores purports to assess only knowledge.

There is no data source for changing attitudes or perceptions. If the curriculum pretest/posttest includes attitude items, these might be used to measure outcomes for attitudes, if those items were separated from the items reporting knowledge.

There are no data sources for behavior in older children. The online Behavior Checklist might be used for this.

The program teaches peer refusal skills, but these are not measured or used to report as outcomes. A simple checksheet or scoring system could identify number of participants who are successful at peer refusal before and after instruction.

Instruments might be designed to evaluate how children under 9 improve on problem-solving skills or academic work.

Prevention Program Theory/Evidence-Based Programs Match

The SMART Moves program is a nationally replicated substance-abuse (and sexual abstinence) program promoted by the Boys and Girls Clubs of America.

However, I have been given little evidence that the curriculum design is evidence-based and science-based. One positive research report was done by the Boys and Girls Clubs of America itself (below). Corroboration by independent sources would be preferred.

SMART Moves is not mentioned by SAMHSA as a model, effective, or promising program.

It is also not mentioned in NIDA’s Preventing Drug Use Among Children and Adolescents, 2nd Edition.

Steven P. Schinke, et al. (1991). The effects of Boys & Girls Clubs on alcohol and other drug use and related problems in public housing projects. New York: Boys & Girls Clubs of America. ED338739.

Calculations of Cost Effectiveness

No evidence was provided on program cost-effectiveness.

ACTION PLAN

Submitted Action Plan

Action Plan

Followup on the Action Plan is suggested.

Also suggested is the development of measured outcomes related to attitude/perception and tracking those outcomes, alongside perhaps the development of new strategies or approaches to affecting attitude/perception.

Direct questions or comments about the Web site to jordan@tempest.coedu.usf.edu

Knowledge about substance abuse addressed?	Yes. Knowledge and attitudes about the dangers, and skills in handling peer pressure. For the younger kids (6-7 year olds), just making them aware it exists.
Attitudes about substance abuse addressed?	Yes. (see above)
Substance abuse behavior addressed?	No.
Refusal skills/peer resistance taught?	Yes.
Negative media messages addressed?	Yes.
Stress management/anger control/self-control addressed?	Yes.
Decision-making skills/critical thinking addressed?	Yes.
Self-esteem addressed?	Yes. Work with self esteem with older kids.
High and positive life goals promoted?	Yes. Start with life goals even with young kids – long-term consequences of decisions with 2nd graders.
Self-efficacy/sense of personal power/personal optimism promoted?	Yes.
Other issues addressed?
Opportunity for youth leadership with peers or in community?	Yes. Leaders in Training at two clubs with other teen leadership through environmental programs.
Recognize program participants for achievement? (e.g., awards, celebrations)	Yes. Banquets for sports events after the season ends. Monthly Youth of the Month Recognition Dinner. Annual Youth of the Year competition.