Opportunities

Evaluation Report
Leo Gafney
October 17, 2012

Summary The administrators of the HCOP have been alert in identifying areas that are working and making adjustments in areas that need improvement. The model itself has been successful because it offers experiential learning as the core of the program, building knowledge and motivation on this foundation. Success factors and improvements that the evaluation has noted include:

• Hiring an administrative staff and instructors with the background and competence to implement the activities successfully;
• Embedding the English, mathematics, and science curricula in the world of allied health and related careers;
• Revising the recruiting process to include more contacts and improved interviewing to identify a wider population of candidates and to select participants more carefully;
• Incorporating contact with a wide range of engaging professionals who are active in different fields of allied health;
• Adjusting the teaching/learning program as needed, for example adding a second mathematics class, to better meet the program’s needs;
• Including practice with interviews, group projects, and presentations;
• Taking advantage of the human and other resources available at Stony Brook Medical Center;
• Creating a tone that is serious and supportive, helping students understand both the opportunities and responsibilities that will be part of their lives in the coming years.

Background The Health Careers Opportunity Program (HCOP) offers a wide range of academic and hands-on experiences related to the training required and careers available in allied health. The program is sponsored by the School of Health Technology and Management and has completed its third year at the Stony Brook University Medical Center. Initial activities have been refined, with adjustments based on findings. This evaluation report is devoted primarily to the HCOP Summer Academy for Long Island area high school students.

Evaluation Method and Activities The evaluation activities were similar to those conducted in the past two years, using methods that included:

• Review of program plans with administrators;
• Surveys of students and parents participating in the orientation program;
• Surveys completed by students after the program;
• An online survey completed by teachers;
• Site visits that included student focus groups and discussions with teachers;
  • Data analysis;
  • Report preparation.

Recruiting and Selection Recruitment for the Academy included expanded contacts to school districts on Long Island, particularly those serving minority students who revealed an interest in the health careers and an aptitude for the sciences. More than 200 students applied and 74 were accepted into the program. These were selected because they were motivated and were likely to profit from the Academy experiences.

Orientation The orientation, intended to provide students and their parents with a clear understanding of the program’s goals and activities, was considered highly successful by both groups, as can be seen in the following table of responses to the orientation survey. Respondents agreed or agreed strongly with all of the items

Table 1. Orientation: Student and Parent Responses  (N = 40)

	Students		Parents
Questions:	Agree Strongly	Agree	Agree Strongly	Agree
The Orientation session was well run	34	4	37	3
I understand the goals of the HCOP summer program	39	1	37	3
I think the activities described will help motivate participants toward health careers	36	4	37	3
There are many opportunities for satisfying health careers	36	4	38	2
I believe that the program will show how English, math and science are useful.	33	7	38	2
I am committed to full participation in the program.	40	0	36	3

Responses to open-ended items about the orientation reflected a high level of interest and engagement by both students and their parents. Parents commented that the orientation was well organized (7) and that they liked everything about it (10). Both groups particularly liked the activities that offered an experiential view of what the program would be like. The students commented positively about the slides and videos of previous years (7) and the speakers (7). Both groups commented on the information gained from the speakers. Eighteen students said they particularly liked the white coat ceremony.

In responding to an orientation survey item asking what is most important in the program, students identified a number of areas: learning about careers in the medical field (12); developing personal and academic skills: dedication, responsibility, self understanding (5); and the opportunity to learn more about college (2). In responding to the same item parents were impressed with the opportunities the students would have (3) in general; and in learning about the medical fields (6).  

Parents were enthusiastic in commenting about the program.

• I am so blessed to have HCOP in my daughter’s life as I know she will do well and enjoy every minute of it!  Thank you!
• My daughter came back as an ambassador.  The program is fabulous, especially since you have expanded it.  I hope the program is available for my other daughter in 2013!
• It is important for students to learn the tasks and responsibilities that will be expected in the health fields.

Structure and Format of the Program The summer Academy is structured so that students will:

1.  Understand both the requirements and the education needed to enter the different health professions;
2.  Gain practical experiences in the health professions;
3.  Increase their base of academic skills needed for college;
   1.  Learn and practice the social and communications skills needed to present themselves effectively in educational and employment situations.

These different student objectives are woven into the format and fabric of the program. The daily schedules include: classroom activities in the core disciplines and SAT prep, talks and demonstrations at different sites within the hospital, hands-on learning, field trips, lectures by health care professionals, team activities, and individual or team student presentations. The Academy takes place within the Stony Brook Medical Center. The students wear the white coats of the medical profession and walk the halls alongside doctors, nurses, and those engaged in a wide variety of the health care issues they are learning.

Program Development The academic structure of the program is guided by several standards. First, the curriculum follows the HRSA requirements and guidelines and in so far as possible embeds the academic skills and concepts in health-career situations. Second, the students learn practical skills that will improve their performance on the SAT exams and in academic classes. Finally, the program includes teaching/learning activities that the students will need as they present and conduct themselves in higher education and in the work environment.

The ongoing development of the program has led to a stronger curriculum and a very positive teaching/learning environment. The mathematics SAT skills were taught not in test-taking format but in the context of problem solving. The program has been adjusted to include two mathematics classes as well as the two English classes that had been part of the program in previous years. Teachers commented positively on the practical aspects of the academic activities.

Everything we did fit; there was a real connection, for example in their math skills; so the goals fit together very well; and we saw a very nice increase from pre to post tests.

Normally English is kind of removed, but here they were excited to tell me what they learned. And they were happy to use what they had learned in interviews.

In science, I had a sense that there were some who wanted more; so I spent more in the labs; we did genetic modification.

We wanted to build up confidence in their writing. Got them writing quickly, not waiting; worked well.

They know that their vocabularies are not where they should be. But they don’t have to know how to define every word. They can figure a lot out from the context. They can use their knowledge of prefixes, etc. to figure some things out. Doing this gives them confidence.

Academic Skills and Pre/Post Measurement The decision to limit the numbers of skills and concepts that would be covered in the different classes—two English, two mathematics, and one science class—again bore positive results, with even more dramatic gains than were achieved in the previous summer. Teachers for each of the five areas prepared pre/post tests and these were administered at the beginning and end of the program. The following table shows the average pre/post test scores and the percentage increase for each, using the pre-program score as a base.

Table 2. Pre/post Scores and Percent Increases by Disciplines

	Pre	Post	% Increase
SAT English A	60.2 (N=24)	98.2 (N=24)	63%
SAT English B	64.2 (N=23)	97.7 (N=23)	52%
SAT English C	55.7 (N=23)	95.8 (N=24)	72%
SAT Math A	34.5 ( N = 25)	54.4 (N = 25	58%
SAT Math B	41.4 (N = 22)	57.7 N = 22)	40%
SAT Math C	35 (N = 24)	52.9 (N = 24	51%
English A	79 (N = 25)	97 (N = 25	23%
English B	75.6 (N = 24)	93.3  (N = 24)	23%
English C	80.2  (N = 23)	93   (N  = 23)	16%
Science A	48.4 (N = 25)	90.4 (N = 23)	86%
Science B	52.5 (N = 24)	82.6 (N = 23)	57%
Science C	55.8 (N = 24)	87.9 (N = 24	57%
Medical Math	57 (N = 74)	92 (N = 740	61%

These increases represent substantial academic progress by the participants and reflect positively on the teaching/learning structure of the program.

 Table 3. Post-Program Student Ratings (N = 70)

For each item, please indicate your agreement or disagreement. (5 = Agree strongly; 4 = Agree; 3 = Neutral; 2 = Disagree; 1 = Disagree strongly)

I believe that the Health Careers Opportunity Program has:	Mean	% 4 or 5
1. Increased my interest in allied health careers.	4.87	100
2. Showed me that there are many different kinds of careers in allied health care.	4.94	100
3. Increased my understanding of what you must do to pursue a career in allied health care.	4.93	100
4.  Showed me that there is a big need for people in health care fields.	4.84	100
5.  Showed me some of the things I might want to study in high school and college.	4.78	100
6. Showed me that English, math and science are useful.	4.84	100
7. Made me more interested in all my school work .	4.58	96

Table 4. Student Interest in Health Fields

I feel that the HCOP Summer Academy has increased my interest in the allied health field of:     (5 = Agree strongly; 4 = Agree; 3 = Neutral; 2 = Disagree; 1 = Disagree strongly)

	Mean	% 4 or 5
a. Anesthesia Technology	3.77	77
b. Athletic Training	4.33	91
c. Clinical Laboratory Science	4.19	88
d. EMT/Paramedic	3.65	73
e. Environmental Health	3.69	73
f. Nuclear Medicine	3.94	77
g. Occupational Therapy	4.32	91
h. Physical Therapy	4.46	91
i. Physician Assistant	4.57	92
j. Radiologic Technology	3.91	77
k. Respiratory Care	4.25	91

Student Response to the Program In addition to the positive reactions to the program that can be seen in the statistics above, students were enthusiastic and articulate in describing the impact of the summer—in answering an open-ended item on the post program survey, and in the focus group.

• They reported dramatic changes in their understanding, interest, and possible pursuit of health careers; and the relevance of academics to career choices.

This program was a big help for me not only because it made me realize that I don’t want to be an accountant, but also because it made me realize that I could put a little more effort into my school work; if I take the time to actually listen, I would understand English, Science, and Math better.  I think I would like to pursue a career in the health field. Before this program, I completely put this career out of my mind.

Experiencing all these allied health professions makes me want to do them all.  My two favorite professions are athletic training and respiratory care.  I’ve learned that no matter where you fall in the health field you will still have fun and enjoy yourself because every profession we’ve learned about was fantastic.

The HCOP summer academy has expanded my knowledge to many different careers in the medical field.  When we went to the school of nursing, I was very interested to the point that I really like the nursing career.  Now I am thinking of that being my career, because I feel this is a great way to help people.

• They talked about a general sense of awakening to their own abilities and possibilities.

The HCOP summer academy was a very valuable experience.  I have learned a lot the past weeks and want to use my knowledge to continue to improve.  Every aspect of HCOP was perfect.  It definitely increased my interest in the allied health professions.  I will definitely finish my last year in high school more confidently.

The HCOP Summer Academy was AMAZING.  I learned so many things that will become key components for my future. It taught me how to save money for college; I have my college essay done, and feel prepared for the college application.  I am so grateful for this program HCOP 2012!

• The students expressed a deeps sense of gratitude for all aspects of the program.

I am truly honored and very thankful for having an opportunity to be part of this outstanding program.  Experiencing the clinical rotations, academics, and the team-building activities have certainly taught me lessons that I will use in the future.  I can go on forever writing about how wonderful HCOP is, and I enjoyed meeting new friends, too!

Thank you to everyone that made this possible.  Everyone was so dedicated to teach us.

In the focus group discussion, the students talked about how the program had given them a fresh understanding of the importance of their academic work and the relevance of various skills. In discussion, it became evident that the students understood and appreciated the different aspects of the program. 

There is more to nursing than I had thought. They might ask you to go in and participate in surgery. And if they tell you to give medicine, you should know the exact dosage. It is a very responsible job.

We learned how to make the essay more personal, using my own background and explaining how that would make me a good candidate. He taught us to write more in depth about exactly who we are; we are not all the same.

You don’t have to be doctor; there are a lot of other careers We learned about the amount of time and  academic requirements for different careers. We got to practice doing needles on rubbery babies. 

I want to be a surgeon but I didn’t understand how the allied health professions help after surgery, for example in physical therapy; there is a lot of teamwork in medicine.

We saw different surgeries on a TV monitor.  We went to the pediatric center; they are creating a more friendly atmosphere for children. … We watched interviews and when we were given background information, we did some role playing about symptoms and diagnosis, learning how to ask questions.

The comments reveal how students changed in thinking about themselves, the importance of education in their lives, and allied health-related careers.

Teacher Experiences and Response The HCOP Academy teachers were interviewed as a group and some of the teachers and administrators completed a survey about the program. Responses to the following items indicate that all of the respondents believed the program was successful (N = 5).

Answer Options	Agree strongly	Agree
The program goals were clearly explained at the outset.	5	0
The overall organization was well conceived.	5	0
The program was implemented smoothly.	5	0
The daily structure worked well.	5	0
The health career activities were well planned.	5	0
The academic portion of the program fit the health career goals.	5	0
Students were generally enthusiastic about the program.	4	1
Students made appropriate academic progress.	4	1
Students grew in their understanding of health careers.	5	0
Students developed an interest in health careers.	5	0
The program goals were generally achieved.	5	0
I would hope to work in the program again.	5	0

The teachers added detail in their written comments, saying that they as well as the students and parents found that the program achieved it objectives in different areas.

It was most telling to hear the students' parents at the closing ceremony praise the program for the changes they saw in their children's academic and professional goals.

The pre/post tests were indicators of what this program has accomplished. It also serves to reinforce the positive feedback that we have received from the university presenters and the students themselves.

Every student wrote a letter to me at the end of the program. Each one wrote about how rewarding the program was and how much they benefitted from it. They loved it.

The program was well thought out and well planned. Dr. Vidal, Erik, Jennifer and Diana Jones were organized, professional, caring and patient. They had clearly stated standards and stuck to them. I wish "school" was like this.

As was described above, the teaching/learning environment for the 2012 summer was highly positive. The evaluation confirmed the presence of teamwork in several program areas: administrators and teachers working together to revise aspects of the program and incorporate an even greater number of engaging activities; students cooperating in team projects and presentations. The program has developed a set of targeted, exploratory learning experiences, increasing students’ knowledge and motivation toward health-related education and careers. The strategies and overall approach offer a model for engaging students, particularly those from under-represented minority backgrounds, in educational and career-related activities that promise to change the direction of their lives.