Steve Schellenberg, Editor

St. Paul Public Schools

1001 Johnson Parkway

St. Paul, MN 55106

In this issue:

NATD President’s Column

          Katrina, Rita, Children in Poverty and Test Directors                       1

The National Council on Measurement in Education (NCME)                     2

          And NATD: A Long-Standing Relationship

Nominations Sought for Certificates of Recognition                                    4

Nominations Sought for NATD Leadership Positions                                 4

Members to Vote on Amendments to the NATD Constitution                     4

National Accessible Reading Assessment Projects                                      5

AERA/NCME/NATD/DRE Meetings – San Francisco, April 5-11, 2006     6

NATD Symposium – San Francisco, April 2006                                        7

          Beyond NCLB: From Measuring Status to Informing Improvement

Report on the August Meeting of the Joint Committee on Testing Practices 8

Contributors to this newsletter included:Peter Hendrickson, James Impara, Thel Kocher, George Olson, Bonnie Wilkerson and Judith Levinson.

NATD President’s Column

Katrina, Rita, Children in Poverty and Test Directors

Peter Hendrickson, Ph.D.

NATD President

As the disasters that were Katrina and Rita continue to unfold, the impacts on school children touch each of us. What if our children, our students had been trapped four days in the New Orleans Convention Center, four days of terror, four days of desperation?

Never before have so many American school children been uprooted and dispersed from their homes, their schools, their cities. As test directors spread across the nation, what should we do after we’ve given from our hearts and our pocket books?

Do we throw in our lot with the Union of Concerned Scientists, seizing the moment on behalf of the poorest, the dispossessed? Perhaps so. If we bring our data to the table about the terrible impacts of poverty, of children bereft of decent health care, of families in continuing crisis, might this generation have a better shot? Perhaps so.

Until then, we might each of us ask four kinds of questions:

1. Have I gone beyond the reporting required by NCLB to speak truth to those in power about the academic and social status of children in poverty? Who among the poorest are advancing and who are not? How might we explain their different paths?

2. Are our own student data secure? If fire, flood, earthquake or plague of locusts were to destroy our workplace, would our records and working files still be accessible? And if they were spared the ravages of a natural disaster, would they be secure if there was a lapse of civil order? Do we have truly remote back-up?

3. If our students were scattered to the winds, would their records be clearly understood by a receiving school? So often I’m asked by a program manager or principal to interpret the records of a new student. The scores and reports received from afar are in metrics and formats whose meaning is lost or obscured outside the sending district or state.

4. Have I joined the community of test directors to improve my own practice and to support the work of my colleagues on behalf of children? Have I moved beyond NATD membership (a bargain at $20 per year) to active participation?

Peter Hendrickson is Assessment Specialist with the Everett (WA) Public Schools and was a shelter manager during the 1990’s floods in Southwest Washington. While an elementary principal in Oberlin, Ohio, he snowshoed through a second story window after a blizzard to shovel out classrooms. And he was a teacher and school board member in Ecuador during a 1972 military coup which closed Academia Cotopaxi and other schools for several days.

Notice in the title that I have characterized the relationship between NCME and the National Association of Test Directors (NATD) as one of long standing. When you get to be my age, you try to avoid the “O” word (old).

Even though the connection between our two organizations goes back to the mid-1980s, there are many NCME members who are not aware of NATD. This is due in part to NCME’s diverse membership. Many NCME members are affiliated with higher education or are employed by testing organizations and work in the “back room” so they are not aware of much of what is happening in the outside world. There may be NATD members who are not members of NCME and who are not aware of the relationship between NATD and NCME and what NCME has to offer to its members. This article tries to provide some information about both of these topics.

The relationship between NCME and NATD

NATD was formed in 1983, growing out of another organization called the Large City Test Directors. It first met as the NATD at the annual conference of NCME in that year. One aspect of the relationship between the two organizations is that since 1984 NCME has provided one of its program slots to NATD. This is a guaranteed slot and the content and speakers are all decided by NATD. In addition to this program slot, NCME has a position on its Board of Directors that requires the Board member to be (at the time they are nominated) working in a K-12 school setting. Although this does not guarantee that the Board member will be an NATD member, it is often the case that this is so (the current person in this slot is NATD member Duncan MacQuarrie). NCME would like very much to increase the involvement of NATD members in NCME activities, particularly in providing articles to our various publications and becoming more involved in presenting both papers and professional development sessions at the annual meeting.

What is NCME

The National Council on Measurement in Education (NCME) is a professional organization for individuals involved in assessment, evaluation, testing, and other aspects of educational measurement. Members are involved in the construction and use of standardized tests; new forms of assessment, including performance-based assessment; program design; and program evaluation.

NCME members include university faculty; test developers; local, state and federal testing and research directors; professional evaluators; testing specialists in business, industry, education, community programs, and other professions; licensure, certification, and credentialing professionals; graduate students from educational, psychological, and other measurement programs; and others involved in testing issues and practices. Our membership is very diverse in terms of where we work and what we do.

What NCME offers its members

All NCME members receive three quarterly publications.

The Journal of Educational Measurement (JEM) publishes articles on technical and theoretical developments in measurement and improvements in the application of measurement methods in educational and other community settings.

Educational Measurement: Issues and Practice (EM:IP), aimed at practitioners and users of tests, publishes articles intended to promote a better understanding of educational measurement and to encourage reasoned debate on current issues of practical importance to educators and the public.

The NCME Newsletter contains announcements and news about current events pertinent to research and practice in educational measurement and also provides NCME members with information about the activities of the NCME Board and committees. There will be article by Peter Hendrickson in an upcoming Newsletter that describes NATD to NCME members. These articles are being written in hopes of bringing members of the two organizations closer together.

In addition to these regularly scheduled publications NCME produces instructional materials that are published as they are completed. These instructional materials are intended to assist individuals who teach educational measurement, program designers and evaluators, and classroom teachers. These materials include references for additional reading. Instructional modules are published in Educational Measurement: Issues and Practice. These materials can be identified by the acronym ITEMS (Instructional Topics in Educational Measurement Series). The ITEMS modules can be downloaded as PDF files from the NCME web site (http://www.ncme.org).

Other Benefits of membership

Networking

NCME meets annually in conjunction with the membership of the American Educational Research Association (AERA) and NATD. The annual meetings offer members of all three organizations opportunities to expand their knowledge and skills and to share their expertise in a convivial atmosphere. Training programs on specific topics, such as classroom testing, are offered during the annual meetings and provide practical information for application in both work and research settings.

Listserv

The General Listserv is intended to foster communications among all NCME members and should be the primary list for interchanges among most NCME members.

Participation in governance

NCME has an extensive committee structure that reflects the interests and activities of membership. Members are involved in NCME committees that include administrative, membership and recognition, publications, standards, and outreach activities.

Job and educational opportunities

NCME’s web site includes an “Opportunities” page that directs those interested to see what jobs are available in a variety of sectors. In addition, this page links to descriptions of higher education programs that offer measurement and evaluation programs in universities across the US, including assistantship opportunities.

I have tried to summarize the relationship between NCME and NATD and to describe briefly a little bit about NCME in hopes that NATD members who are not current members of NCME will consider joining. A membership application can be downloaded from the NCME web site. You can also learn a lot more about NCME from that web site. (http://www.ncme.org)

NATD is represented on the General Advisory Committee to these projects. Originally funded by the Office of Special Education Programs, federal responsibility for the two projects has been transferred to a newly established division on special education research housed within the Institute for Education Sciences.

The two projects that are currently funded to conduct research on accessibility are Designing Accessible Reading Assessments (DARA) based at Educational Testing Service and Partnership for Accessible Reading Assessments (PARA). PARA is a consortium of the National Center on Educational Outcomes; the Center for Research on Evaluation, Standards, and Student Testing; and Westat, Inc.

Much of the initial work of the projects has focused on a collaborative effort to formulate a definition of reading proficiency that can be used as a basis for research and development for accessible large-scale tests of reading. NATD members that attended AREA/NCME or the CCSSO Large Scale Assessment Conference, received invitations to provide input into this definition through focus groups that were held in conjunction with these two meetings. Focus groups were also conducted at meetings of the Council for Exceptional Children, the International Reading Association, and the Society for the Scientific Study of Reading. The staffs of the two projects along with the federal project monitors continue their efforts to refine the definition.

Another major focus has been the development of a set of disability reports. These reports provide general information about specific disabilities and how they interact with reading, so that reading professionals and others who might contribute to the development of accessible reading assessments understand some of the challenges that need to be addressed. Seven papers have been developed and drafts are being circulated for comment before the papers are finalized.

Both projects have also been engaged in empirical research. PARA has engaged in distractor analysis of existing state assessment data and DARA has conducted DIF studies and factor analyses of existing state assessment data. DARA is collecting data for a large experimentally designed differential boost study. This study will look at the effect of audio presentation as compared to print reading only.

More information is available at the joint NARAP (National Accessible Reading Assessment Projects) website www.narap.info and at the individual project websites

www.ets.org/dara and www.readingassessment.info.

AERA/NCME/NATD/DRE meetings

San Francisco – April 6-11, 2006

The table below reflects our current understanding of dates, times and venues for AERA, NCME, NATD and DRE.

Thurs., Apr. 6 all day DRE (Directors of Research and Evaluation) meeting location TBA

Fri., Apr. 7 a.m. NATD breakfast location TBA

NATD business meeting

noon AERA meeting begins Moscone Convention Center

evening NATD Board meeting

Sat., Apr. 8 all day NCME meeting begins Hotel Nikko

(tentative) NATD Symposium – see facing page for details Hotel Nikko

evening NATD Dinner hosted by test publishers location TBA

Sun., Apr. 9 a.m. NCME breakfast location TBA

Mon., Apr. 10 a.m. AERA Division H breakfast location TBA

all day last day of NCME meeting Hotel Nikko

evening Division H Social location TBA

Tues., Apr. 11 all day last day of AERA meeting Moscone

A more detailed schedule will appear in the spring newsletter

NATD Symposium – San Francisco, April 2006

Beyond NCLB: From Measuring Status to Informing Improvement

While recent focus on large scale assessment has been in the realm of accountability to benchmarks and standards, important issues of how movement toward those same benchmarks and standards can be facilitated through large scale assessment need to be addressed. This symposium will focus on identified issues of note in using data to move beyond the accountability focus of NCLB and toward a focus on improving student learning.

Although the issues of data use to improve student learning are legion and complex, they can be rather simplistically described in three contexts: having the right data, having the tools to help translate data into meaningful instructional information, and having the knowledge of the data and data tools to be able to use them together.

Presenters:

Eva L. Baker, University of California –Los Angeles

Title: Turning an Accountability Policy into a Learning System

G. Gage Kingsbury, Director of Research, Northwest Evaluation Association

Title: Tools to Translate Data into Meaningful Instructional Information

Judith A. Arter, Assessment Training Institute

Title: What Users Need to Know and Be Able To Do To Use Data for the Improvement of Instruction: a perspective on bringing teachers and student-learners into the analysis of test results and planning of future actions.

Discussant: Karen E. Banks, Consultant, Data Detectives

Moderator: Bonnie C. Wilkerson, Northbrook School District 27, Northbrook, Illinois

The National Association of Test Directors is a member of the Joint Committee on Testing Practices (JCTP). Two members of NATD (Judith Levinson and Stephanie Gertz) represent our testing organization at bi-annual meetings. The most recent meeting was held in late August in Washington D.C. hosted by the American Psychological Association. The following organizations were represented:

· American Psychological Association

· American Speech-Language-Hearing Association

· National Council on Measurement in Education

· American Educational Research Association

· American Counseling Association

· National Association of Test Directors

Topics at the August meeting included:

ABC’s of School Testing. Revision of the ABC’s Leader’s Guide is in process. The working group will determine the extent of the necessary revisions, as well as locate references to other existing resources of similar content.

Bylaws/Policies and Procedures. The group is beginning a project to translate the JCTP bylaws into a policies and procedures manual.

Distribution of the Code of Fair Test Practices. The Code is being distributed to test-related organizations, testing companies and school districts. Testing organizations will be contacted for endorsement of the Code. The Code can be downloaded by going to the following website:

http://www.apa.org/science/FinalCode.pdf

The next meeting of JCTP will be in April in conjunction with the AERA annual meeting.

Peter Hendrickson, President	Philip Morse, At Large
Bonnie Wilkerson, Vice-President	Mike Flicek, At Large
Thel Kocher, Past President	George Olson, At Large
Steve Schellenberg, Secretary	Mike Strozeski, At Large
Sherry Rose-Bond, Treasurer	Ray Fenton, Webmaster