Appendix Guide

INTRODUCTION:

Data derived from assessments can be interpreted in a number of ways.  At the most basic level, group means, standard deviation and standard error of the measure (Appendix A: group means, standard deviation, standard error of measure) may be used to interpret individual test scores.  Scores falling between +1.00 and –1.00 standard deviations, or between the 16th and 84th percentile are considered to fall within normal limits.  While students obtaining scores within the lower bounds of this range may be experiencing substantial academic difficulties, it is important to note that these difficulties would be considered fairly common for the school population as a whole.

While a student’s score on any particular measure may or may not fall within normal limits (+ 1.00 SD), it is important to keep in mind the reliability of the assessment tool and the error associated with that reliability.  Along with examining where the student’s obtained score falls within the normal distribution, it is important to also consider where the student’s true score lays (Appendix A: Confidence Interval) within this range.

Introduction:  Instructional Event # 4:

The next assessment tool that is presented does not use standard deviations or means to look at student progress.  It is an individualized rating system of assessment that tracks the progress of students with more significant disabilities from preschool through exit from high school.  Using the following activities allows you to see the need and validity of the different measures that have been presented and the data they present.

Instructional Event # 4:

Outcome:        Given Indiana Standards Tool for Alternate Reporting (ISTAR) data on an individual student, the participant will evaluate the current performance and progress of the student and identify strengths and needs.

Narrative Introduction:

The Indiana Standards Tool for Alternate Reporting (ISTAR) is a standards-based, web based, assessment system provided by the Indiana Department of Education.  Federal law requires that all students participate in the state’s assessment system.  ISTEP+ is the assessment used for the majority of students in the state.  For a small number of students with disabilities, however, the ISTEP+ is not an appropriate tool.  For those students, the teacher rated ISTAR system is used.  ISTAR is the only alternate assessment recognized by the state of Indiana, and is intended to be a tool for continuous progress monitoring for students with disabilities.  The ISTAR is required for:  all early childhood students in programs for children with disabilities and all students with significant cognitive disabilities in grades 3-10 who meet specific participation criteria (as an alternate assessment in lieu of ISTEP+).  In addition ISTAR has proven to be an additional supplemental evaluation tool for a variety of students with and without learning difficulties.  

It is important that all building level administrators have an understanding of the ISTAR system, eligibility requirements, and student outcome data.  ISTAR assessments DO count toward participation rates as well as adequate yearly progress.  The following activities are intended to support your understanding of the ISTAR data and its potential uses as an assessment tool.

1. Go to the ISTAR website at www.istar.doe.state.in.us.  Click on Training Materials at the top of the page, then click ISTAR/Case Conference, and finally ISTAR Handbook on Alternate and Supplemental Assessment
   
    
2. Print a copy of the Handbook (the complete handbook is 38 pages long)
   
    
3. Read the information through page 21.  Highlight information that is new to you or that you feel is pertinent to your position. 

It is not only important to have either a new or reviewed understanding of the system and its intent, but what the gathered information is telling you. 

1. Review the information on pages 16-18 of the ISTAR Handbook (VI.  Scoring and Reporting and VII Use of Results)
   
    
2. Print out the following report examples that are available for assessed student
   1. ISTAR Summary Report (Appendix F)
   2. ISTAR Functional Summary Report (Appendix G)
   3. ISTAR Assessment Detail Report (Appendix H)
   4. ISTAR Functional Detail Report (Appendix I)

The available reports on the ISTAR system can look at student data at the present time, or look at progress from one rating time to the next.  The summary reports you have been provided are comparison reports, showing past data along with present.  It is important to not underestimate the importance of this information.  This information has the ability to drive the goals and objectives for a student, as well as assist administrators in reviewing curriculum and standards that are being used for participating students.  In addition, when used as a supplemental assessment, ISTAR data provides documentation that compliments ISTEP+ as well as locally collected data.

ISTAR Report Information (reference pages 16-18 in the ISTAR Handbook):

ISTAR Summary Report (Appendix F): 

The summary report provides a one page student graph that provides total scores on the standards in the areas of Mathematic, Language Arts, and Functional Achievement.  As explained in the handbook (page 17) the Math and Language Arts scores are based on a Basic (ages 2-3) through grade 10 level.  The age or grade equivalent is the number on the left side; while the right side number indicates the percentage of that level that the student achieved (for example:  1.24 would mean that the student has demonstrated 24% of the 1st grade skills for the given assessed area).  The Functional data reflects the percent of independence with which the student demonstrates the skill, based on 100%.  Since this is a comparison report, there are two bar lines.  The darker bar line represents the current assessment.  The lighter line indicates the assessment data from a previous assessment (in this case, 2-1-07).

Justin (Guided Sample)

1. Math Performance:  The sample ISTAR Summary Report that you have been provided indicates that this 10th grade student has achieved a 2.71 on overall mathematics performance.  This means that he has demonstrated 71 % of the 2nd grade math indicators.  To look more specifically at mathematic skills, the subsets that were rated are listed below.  Each subset, again, indicates the performance based on % of grade level indicators that have been demonstrated.  In the area of Number Sense, Justin achieved 32% of the 3rd grade indicators on 2-1-07.  The current assessment (10-31-07) indicates that he demonstrates 45% of the 3rd grade indicators.  Therefore, he has improved 13% of a grade level.  On the far right of the Summary Report is the progress ratio for the two compared assessments.  This allows a quick glance of progress (or regression) made by students. 

2. Language Arts Performance:  As in the Mathematics section, the overall Language Arts current level shows that the student has demonstrated 23% of the 3rd grade indicators.  Look at the Writing Convention subset.  This subset shows a significant regression from 6.22 to 4.30.  It is important to look further into such discrepant scores and ask yourself some questions:
   1. Was there a different rater for this assessment, who might have interpreted data differently?
   2. Was the baseline set correctly before assessments were completed? (before assessing skills, the rater answers a number of broad questions in each area to set an appropriate level of assessment)

If true regression is found, goals and objectives should reflect more leaning activities and opportunities for the student to practice and demonstrate the skill sets.

3. Functional:  The data presented is different for this subsection of the ISTAR Assessment.  Instead of looking at grade level indicators and percentages achieved, the Functional section looks at the percentile of independence that the student displays.  For the sample student, this student has achieved a 79% combined percentage of independence within the functional skills domain.  Functional Skills is broken down into:  physical skills, personal care skills, social-emotional skills, and independent living skills.  In this section it is important to keep the student age and grade level in mind.  For example:  you would not expect a 3rd grader to have demonstrated 100% of independent living skills because that is not age appropriate.  For the sample student, he has achieved 100% of physical skills and 100% of personal care skills.  Though progress has been made in both the social-emotional and independent living sections, those would be areas to concentrate on, especially since the student is in 10th grade.  

The ISTAR Summary Report gives you a snapshot look at a students overall achievement of the given indicators.  This information allows teachers and administrators to quickly find general areas of student strengths and needs.

ISTAR Functional Summary Report (Appendix G)

The ISTAR Functional Summary Report further breaks down the data that was presented under the Functional section of the Summary Report.  Broad areas that were rated with the four subsets (physical skills, personal care skills, social-emotional skills, and independent living skills) are presented.  The cumulative percentages of the subsets are what make up the overall percentage for the broader skills.  On this section it is important to remember that the independence that the assessment is looking for is 100% upon completion of high school.  Therefore, some indicators such as preparing food and demonstrating vocational interests may not be age appropriate as of yet so it would be expected that the percentile would be low or perhaps not developed at all.

Justin (Guided Sample)

1. Physical Skills:  The sample student has demonstrated 100% of independence in physical skills.  Subsets within the Physical Skills section include:  demonstrates sensory responses, demonstrates reaching, grasping and object control, demonstrates precision fine motor skills, demonstrates strength, stability and control with locomotive movements, and integrates physical skills.  As was discussed before, it is important to look at each skills set and the age appropriateness of the indicators.  Since we are looking at assessments for students identified with a disability it is also important to keep in mind the disability area and its impact on both the academic and functional summaries. 

2. Personal Care Skills:  As with the Physical Skills subset, personal care skills are broken down into smaller skill sets that have been rated.  As you can see on the sample Functional Summary Report, personal care skills range from demonstrating basic drinking and eating skills to preparing food.

3. Social-Emotional Skills:  Social-emotional skills are broken down into three subsets:  practices interpersonal skills, practices intrapersonal skills, and practices habits of learning.  On Justin’s sample report he has shown increases in each of these areas since the previous rating.  His skills in practicing intrapersonal skills has grown from 50% independence to 56% independence.

4. Independent Living Skills:  The skill sets in this area begin with the basics of stating personal information, to much higher levels of performance including participating in community and leisure activities and demonstrating vocational interests and skills.  Though the sample student appears quite independent in the areas of physical skills and personal care skills, there are many areas he has not demonstrated in the area of independent living.  As you can see by looking at uses common tools and devices in real world applications, last years assessment and this years show a huge percentage of growth.  As discussed earlier, when this happens it behooves the rater to look again at either the baseline assessments (which sets the appropriate assessment level) or rater differences. 

ISTAR Assessment Detail Report (Appendix H):

As an educator, parent, or administrator sometimes the broad picture does not provide enough information.  To drill down to specific skills under each subset, you can access the ISTAR Assessment Detail Report.  This report allows you to view each of the indicators that were rated by the teacher.  Within the Assessment Report, the most recent assessment, all assessments, or assessments between two identified dates can be accessed (refer to page 35 in the ISTAR Handbook).

The ISTAR Assessment Report provides specific information that was rated by the assessing teacher.  On the far right the rater has determined, using a rubric, that the student has either demonstrated the skill, is developing the skill, or that the skill is not evident. 

Justin (Guided Sample)

The guided sample (Appendix H) includes one page of academic assessment indicators for you to view.  As you look at this information, you can easily use the Summary Report (Appendix F) to go from big picture to the breakdown.  For example, under Mathematics, Number Sense, the student (overall) demonstrated 45% of the 3rd grade skills that were assessed.  To look more specifically at what types of questions were asked of the rater, the detail report can be used.  For this student you can see that there are some demonstrated, developing, and not evident skills.  This information is very useful for teachers in identifying goals and benchmarks.  Through the rating rubric, strengths and weaknesses are identifiable that can be used as present levels of performance as well as needed areas for instruction and growth.

Functional Summary Report (Appendix I):  The Functional Summary Report further breaks down the reflected percentage on the Functional section of the Summary Report (which was an overall 69%).  Within each of the four areas indicators have been provided for the teacher to assess. 

Justin (Guided Sample)
The sample ISTAR Summary Report (Appendix F) shows that this student has demonstrated an overall 79% independence for functional skills.  Looking at Appendix I (Functional Detail Report) you can see that under the broad category of social emotional skills (overall 69% independence), this student has demonstrated 79% independence within Practicing Interpersonal Skills.  The Functional Detail Report allows you to view the rating indicators and how they were assessed.  For this student, rated skills were either developing or demonstrated.  Teachers should be using this information to help develop appropriate goals and objectives and teaching strategies. 

Anne (ISTAR Report Application)
Print a copy of Appendix J and Appendix K.  These will provide you with copies of a 3rd grade student’s Summary Report, Functional Summary Report and both Detailed Reports.  Using these, do the following application exercises:

1. ISTAR Summary Report (Appendix J):  Determine present levels of each broad category and subcategories.  Which of the subsets show student growth from 3/23/07 to the present?  At what rate is this student progressing?  Are there any comparisons that require further investigation or discussion?

2. ISTAR Functional Summary Report (Appendix J):  What was the overall percentage of independence indicated on the Summary Report?  What subset of skills does this student show the greatest strength?  Which is the weakest?  Determine if any of the subsets are possibly not appropriate yet due to the students age and or level of ability.  Using only this information, what areas appear to be good targets for goals and objectives?

3. ISTAR Assessment Detail Report (Appendix K):  In looking at the Assessment Detail Report, what skills appear to be developing? Demonstrated? Not Evident?  Looking back at the Summary Report (Appendix J), did this student make progress in each of the areas under Mathematics?  What was the overall progress ratio in the broad area?  How would you recommend that a teacher take the information on the Assessment Detail Report and use it to plan for student instruction?

4. ISTAR Functional Detail Report (Appendix K):  Looking at the Functional Detail Report and the Summary Report, compare the current skills with the previous assessment.  What does this information tell you?  Would this student show growth through monitoring of this assessment?

Summary:
The ISTAR data that is collected can be utilized in a variety of ways.  The Office of Special Education and the Indiana Department of Education each utilize the collected data to look at the overall outcomes of students identified with more significant disabilities.  The data is also used to assist in calculating Adequate Yearly Progress (AYP) for schools and corporations.  This information alone should be important to building level administrators.  Not only is the expectation that students with typical ability make progress yearly, but that ALL students make progress yearly. 

In addition, the ISTAR data should not only help teachers design Individual Education Plans (IEPs), it should assist teachers in reviewing their instruction and curricular materials.  As a building level administrator, this information should be available to you, parents, and collaborating educators so appropriate programming can be made available for students.