The North Dakota PFS Youth Survey is part of the evaluation of a large scale prevention initiative called the North Dakota Partnerships for Success (ND PFS) project from the Substance Abuse and Mental Health Services Administration (SAMHSA). This federal grant was awarded to the North Dakota Department of Human Services (ND DHS) to prevent underage alcohol use. ND DHS selected eleven separate communities based on their demonstrated need with regard to underage alcohol use, and its associated consequences.
The North Dakota PFS Youth Survey will be used in the ND PFS evaluation to track how alcohol use and related attitudes in the selected communities change through the course of the project. Specifically, the survey will monitor the percent of students that have used or are currently using alcohol, and/or abusing prescription drugs. It also measures a variety of attitudes, beliefs, and perceptions related to alcohol and prescription drug abuse. The data will also be used to report the effectiveness of the ND PFS to the federal cross-site evaluators.
This is a list of Frequently Asked Questions about ND PFS Youth Survey. This document addresses the most common concerns from schools and school districts as the consider participating in the ND PFS Youth Survey, and also as they administer the survey.
FAQ for Parents
Why is the survey being done?
The North Dakota Department of Human Services will use ND PFS Youth Survey to measure youth alcohol use and the abuse of prescription drugs for a large scale underage alcohol prevention project. The survey results will be used by communities and the state to help understand the factors that might influence whether or not students use alcohol and/or abuse prescription drugs. Information from this survey will help plan and evaluate substance abuse prevention efforts at both the local and state levels.
How long does it take to fill out the survey?
The survey takes about 10 minutes for most students, but about 15 minutes for 6th grade students. In past years, most if not all, students were able to finish the survey in the allotted time.
Is this going to be a lot of work for me?
No. We are trying to make sure that the survey administration is as easy and simple as possible. We will be doing the work of sorting, organizing and preparing the surveys for all the classrooms and schools where it will be administered. We will be as accommodating as possible to make this process easy for you and will strive to come up with solutions to any concerns.
Who supports this survey?
State level support for the ND PFS Youth Survey comes from the North Dakota Department of Human services. This survey satisfies the substance use and attitude measures that are required for the North Dakota Partnerships for Success project, funded by the federal Substance Abuse and Mental Health Services Administration (SAMHSA). The communities funded by the ND PFS support the survey to help understand and evaluate the prevention strategies that they are implementing as part of the ND PFS. Many local communities, school districts, and schools also support the ND PFS because it provides them with local level information that is not available from any other source to help them design and implement their own substance abuse prevention systems.
Are sensitive questions asked?
Yes, some questions on the ND PFS Youth Survey are sensitive. The primary purpose of this survey is to measure substance use by our youth and related attitudes, beliefs, and behaviors that have been shown to be related to youth use substances. These topics can be considered sensitive. As a result, the questions are written in a direct but non-threatening way. Additionally, the students are informed that their participation in the survey is voluntary and anonymous. The students may skip any questions that they do not feel comfortable answering and their responses are not individually identifiable. Consequently, the risk of responding to the survey is minimal, and is no greater than what the students might experience as they complete their typical school work.
Do students tell the truth?
In general, yes. (See “Do Students Tell the Truth on the ND PFS Youth Survey?”) To summarize:
Reliability and validity. Based on published studies of the psychometric properties of these survey items, the reliability and validity of these measures do not seem to be compromised because the students are not telling the truth. If you would like to read more, WYSAC can give you copies of these published papers.
Survey environment. The survey is conducted in a way that reassures the students to feel that they can answer the questions openly and honestly because their survey responses are anonymous.
Consistency across time and with other survey results. Though the ND PFS Youth Survey is new, it is consistent with other youth surveys like the Youth Risk Behavior Survey and Monitoring the Future. Given these similarities, the ND PFS evaluation team, who have conducted similar youth surveys in the past, expects that the ND PFS Youth Survey will be consistent with other national, state and local surveys. Thus, the results are expected to be consistent and comparable both across time and across surveys.
How do parents find out about the survey?
The primary means of informing parents about the survey will be through a letter sent to parents. The letter will be distributed approximately two weeks before the school’s survey administration date. WYSAC will provide the letter that will be sent to the parents at each school. If so desired, schools may also inform parents about the survey in other ways, such as newsletters. WYSAC will work with the schools to ensure that all parents are informed of the survey prior to the survey administration date.
What if some parents do not want their children to complete the survey?
All parents have the option to request that their child not take the survey. This process will be explained during the survey administrators’ training.
Will students’ names be used or linked to the surveys?
No. The survey has been designed to protect the students’ privacy. Students do not put their name on the survey. When students finish the survey, they place their questionnaires in a collection box or envelope with all the other students’ questionnaires. The surveys will not be individually identifiable after they are placed in the collection box.
What data do we receive if we participate in the ND PFS Youth Survey?
Reports based on the responses provided by your students will be written for the state, community coalitions working on the prevention of underage alcohol use, and school districts. Individual schools may also request a report if they so desire. If a school or district requests a report, then a report will only be written if a sufficient number of students at that school participated in the survey to ensure that the students’ anonymity and confidentiality are not breached.
WYSAC may compare results from county, school district, and school reports with the state and, when appropriate, national results. Additionally, data will be presented to compare results across time. WYSAC will consult with a wide variety of communities, schools, and school districts, along with people from the state to determine what and how the information should be presented to make the data analysis clear, understandable, and approachable.
What data will be available to the public?
Data reports for the PFS community service area, and state will be released to the public. These reports will combine student responses across schools and school districts to ensure that a school district or school will not be singled out. School district reports will only be released to the school district superintendent. If a school requests a report, and it is appropriate to write one, then that school level report will only be released to the principal of that school. We will only release a local school district’s or school’s data to other parties with written permission from the established authority within the respective school district or school.
How were our students chosen to be in the survey?
The goal of this project is to survey all middle school and high school grade students in the ND PFS community service areas. Your school was chosen because you teach students who are in one or more of those grade levels and your school serves students who live in a PFS funded community.
Which of my students should I administer the survey to?
The survey was designed to be administered to all students in 6th through 12th grades.
When will the survey be administered?
The survey will be administered in the spring semester. A similar time frame will occur annually through 2020, which is the end of the ND PFS project. The survey can be administered on a Tuesday, Wednesday, or Thursday. We will work with schools to choose the class period on the chosen day for survey administration.
Who can serve as survey administrators?
In most schools the classroom teachers will serve as survey administrators. They will be charged with sending the parental consent letters home two weeks prior to the survey administration date, along with administering the surveys to their own students using the prepared survey packets for each classroom. Each school principal will designate a school survey coordinator to aid the teachers and help assure that the survey process goes smoothly. The school survey coordinator will be responsible for:
- Distributing the parental consent letters to the classroom teachers so that the teachers can send the letters home with the students.
- Distributing the classroom survey packets to the teachers prior to the survey administration date.
- Collecting the completed classroom survey packets and returning them to WYSAC using the prepaid mailing label.
What is the survey administrator training, and when will it take place?
The survey administrator training is a set of ">four short videos that take no more than 10 minutes to complete. Survey administrators can view in the training using any internet-connected computer with access to YouTube.
Where can I get more information about the ND PFS Youth Survey?
The ND PFS Youth Survey website contains a large number of informational resources. The website address is https://NDPFSsurvey.org.
For additional questions, contact the WYSAC evaluation team.
Do Students Tell the Truth on the ND PFS Youth Survey?
The honesty of survey respondents is always a concern in survey research. The ND PFS Youth Survey uses a variety of procedures to encourage students to respond honestly. In addition, there are procedures in place to help detect and correct for some dishonesty in survey responses. Based on these procedures as well as other evidence explained below, it appears that the vast majority of students answer the ND PFS Youth Survey openly and honestly. As a result, we are confident that the data from the ND PFS Youth Survwy can be relied upon as an accurate picture of the Communities' students.
Procedures to Encourage Honesty
The survey administration procedures are designed to foster an environment in which students feel that they can respond honestly. The procedures promote the anonymity of students’ responses and the confidentiality of the schools’ data. The instructions given to the students encourage open and honest responses.
The procedures include:
- Assigning the students to sit as far apart as possible throughout the classroom.
- Restricting the teachers and/or survey administrators to remain in a location in the classroom from which they cannot observe the students’ responses.
- For paper survey administration, designate a central collection box or envelope in the classroom where the students themselves place the surveys. Teachers inform the students that the collection box or envelope will be sealed immediately at the end of the surveying period so that no one in the school can gain access and associate any questionnaire with a specific person.
- For electronic survey administration, a common survey access code use by the entire class that only is associated with the school where the survey is being administered. Additionally, the survey only collects the students’ responses to the survey. Other information that might be personally identifiable, such as the IP address is not collected.
- Telling the students about the importance of providing honest answers. Teachers will tell students that their answers will help improve programs and policies for students.
Questionnaire Design and Content
- The NDPFS Youth Survey questionnaire is designed to protect the anonymity of the students.
- No names or other types of personally identifying information are ever requested. Students are instructed not to put any identifying marks on the questionnaire.
- No skip patterns are used on the questionnaire (e.g., “if you answer ‘yes’ to question 7, skip to question 105”). Every student is asked to answer all the questions. This helps to ensure that all the students complete the questionnaire in approximately the same amount of time.
- To help students accurately understand and respond to the survey questions, the questionnaire is written for a 7th grade reading level. Questions are written in a straightforward and direct manner. For clarity, nearly all the survey questions require the student to choose only one response.
- The survey has been tested to make sure that it can be completed in a single classroom period.
Evidence for Honest Responding
Logic within Groups of Questions
Questions with similar topics often have a logical relationship that would be expected if the students were responding honestly to the survey. For example, more students would be expected to admit to having tried cigarettes during their lifetime than the number of students who have used cigarettes during the past thirty days, and even fewer students would be expected to admit having smoked more than half a pack of cigarettes per day during the last 30 days. In almost every case, survey results validate these logical patterns.
Comparison of ND PFS Youth Survey Results with Results from Other Surveys
When survey results are compared to results from other national, state, and local surveys on the same topics, the results are generally quite similar. This is especially evident when differences in survey administration, sample selection, question-wording, and analysis methods are taken into consideration.
In 2002, a set of researchers from the University of Washington published a scientific paper explaining the development and psychometric testing of the core survey questions used in the survey (Arthur, Hawkins, Pollard, Catalano & Baglioni, 2002). The development included using and modifying established survey measures for key attitudes, beliefs, and behaviors that have been shown in scientific literature to increase or decrease the odds that youth will use alcohol, tobacco, and other drugs or exhibit other problem behaviors.
These survey questions were tested and refined until a survey instrument was developed that could be administered in a single 45 minute class period to 6th, 8th, 10th, and 12th-grade students. The survey questions demonstrated adequate reliability and validity with regard to predicting youth substance use. Further studies (Glaser, Van Horn, Arthur, Hawkins & Catalano, 2005) have found that the core survey questions seem to measure similar attitudinal and behavioral constructs across five different race-and-ethnic groups and for both genders. Overall, these studies indicate that students respond to the survey with enough honesty that it does not generally threaten the reliability and validity of the survey results.
Arthur, M. W., Hawkins, J. D., Pollard, J. A., Catalano, R. F., & Baglioni, A. J. (2002). Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review, 26(6), 575-601.
Glaser, R. R., Van Horn, M. L., Arthur, M. W., Hawkins, J. D., & Catalano, R. F. (2005). Measurement Properties of the Communities That Care Youth Survey Across Demographic Groups. Journal of Quantitative Psychology, 21(1), 73-102.