Item Generation for Test Development 1st Edition Sidney H. Irvine instant download
Item Generation for Test Development 1st Edition Sidney H. Irvine instant download
https://ebookname.com/product/item-generation-for-test-
development-1st-edition-sidney-h-irvine/
https://ebookname.com/product/test-signal-generation-for-service-
diagnosis-based-on-local-structure-graphs-1st-edition-michael-
ungermann/
https://ebookname.com/product/test-driven-development-for-
embedded-c-1st-edition-james-w-grenning/
https://ebookname.com/product/assembly-language-for-intel-based-
computers-4th-edition-kip-r-irvine/
https://ebookname.com/product/the-roman-empire-
divided-400-700-ad-2nd-edition-john-moorhead/
Martin Luther King Jr Penguin Lives First Edition Frady
https://ebookname.com/product/martin-luther-king-jr-penguin-
lives-first-edition-frady/
https://ebookname.com/product/a-sport-loving-society-victorian-
and-edwardian-middle-class-england-at-play-sport-in-the-global-
society-46-1st-edition-j-a-mangan/
https://ebookname.com/product/careers-in-homeland-security-1st-
edition-adam-woog/
https://ebookname.com/product/writing-successful-science-
proposals-second-edition-andrew-j-friedland/
https://ebookname.com/product/prosperity-1st-edition-jane-golley/
Mastering Autodesk Inventor 2014 and Autodesk Inventor
LT 2014 Autodesk Official Press 1st Edition Curtis
Waguespack
https://ebookname.com/product/mastering-autodesk-
inventor-2014-and-autodesk-inventor-lt-2014-autodesk-official-
press-1st-edition-curtis-waguespack/
Item Generation
for
Test Development
This Page Intentionally Left Blank
Item Generation
for
Test Development
Edited by
Sidney H. Irvine
University of Plymouth
Patrick C. Kyllonen
Air Force Research Laboratory and
EducationalTestingService
ISBN: 0-8058-3441-9
Samuel J. Messick
vii
This Page Intentionally Left Blank
Contents
Contributors vii
...
Foreword x111
Henry Braun
Item Generationfor Test Development: An Introduction xv
Sidney H . Imine
Acknowledgments xxvii
Prologue and Epilogue: Remembering SamuelJ. Messick xxix
ChapterOne
The Foundationsof Item Generationfor Mass Testing 3
Sidney H. Imine
Chapter Two
Using the Psychologyof Reasoning to Predict the Difficulty 35
of Analytical Reasoning Problems
Stephen Newstead, Peter Bradon, Simon Handley, Jonathan Evans,
and Ian Dennis
ChapterThree
Approaches to Modeling Item-Generative Tests 53
Ian Dennis, Simon Handley, Peter Bradon,
Jonathan Evans,and Stephen Newstead
Discussant Remarks 73
Len Swanson
Discussion 77
ix
Contents
Contributors vii
...
Foreword x111
Henry Braun
Item Generationfor Test Development: An Introduction xv
Sidney H . Imine
Acknowledgments xxvii
Prologue and Epilogue: Remembering SamuelJ. Messick xxix
ChapterOne
The Foundationsof Item Generationfor Mass Testing 3
Sidney H. Imine
Chapter Two
Using the Psychologyof Reasoning to Predict the Difficulty 35
of Analytical Reasoning Problems
Stephen Newstead, Peter Bradon, Simon Handley, Jonathan Evans,
and Ian Dennis
ChapterThree
Approaches to Modeling Item-Generative Tests 53
Ian Dennis, Simon Handley, Peter Bradon,
Jonathan Evans,and Stephen Newstead
Discussant Remarks 73
Len Swanson
Discussion 77
ix
x Contents
ChapterFour
On the Roles of Task Model Variables in Assessment Design 97
Robert J. Mislevy, Linda S. Steinberg, and
Russell G. Almond
Chapter Five
Modeling the Difficultyof Quantitative Reasoning Items: I29
Implications for Item Generation
Mary K. Enright and Kathleen M.Sheehan
Chapter Six
Item-Generation
Models
for
Higher
Order
Cognitive
Functions
159
Lutz F. Hornke
Discussion 187
Chapter Seven
Generative Testing: From Conception
to Implementation 199
Isaac I. Bejar
Chapter Eight
Generating
Abstract
Reasoning
Items
With
Cognitive
Theory
219
Susan E, Enlbretson
Chapter Nine
Item Generation for Repeated Testing
of Human Performance 25 1
Patrick C. Kyllonen
ChapterTen
Scoring TestsWhen Items HaveBeen Generated 277
David Wright
Contents xi
Chapter Eleven
On the Automatic Generationof Test Items: Some Whens, 287
Whys, andHows
Howard Wainer
PARTIV:APPLICATIONS OF ITEM-GENERATIVE
PRINCIPLES
Chapter Twelve
The MICROPAT Pilot Selection Battery: Applications 317
of Generative Techniques for Item-Based and Task-
Based Tests
Dave Bartram
ChapterThirteen
On the Implementationof Item-Generation Principles 339
for the Designof Aptitude Testingin Aviation
Klaus-Martin Goeters and B e d Lorenz
ChapterFourteen
of Schema Theory
Item Generation and Beyond: Applications 36 1
to Mathematics Assessment
Mark K. Singley and Randy E. Bennett
Discussion 39 1
AuthorIndex 399
Aboutthe Editors 41 1
This Page Intentionally Left Blank
Foreword
...
x111
xiv Foreword
Henry Braun
Educational Testing Service
Item Generation for Test
Development: An Introduction
Sidney H. Irvine
Universip of pbmotltb
XV
xvi Irvine
reliable and robust latent structures and regression equations that can stand
up to validtty replications in all lunds ofdtfferentcontexts, cultural or
dtspositional. Cronbach's goal of a unified main line approach is attainable
in the development of item-generative tests.
How work in item generation has fostered a unified dtscipline of
psychology is a major feature of this volume. Readers wdl find
o v e r w h e h n g evidence of confluence in the pioneeringitem-generation
modelsof authors Bartram, Bejar, Goeters,Hornke,and Irvine, and in
second-generation developments characterized in the chapters by Dennis,
Embretson, Kyllonen, and Newstead; but not confined to their work as one
observes later in this introduction.
Perhaps the obvious and main departure from traditional test
construction, which may be described as a technology with an unendtng
sequence of empirical questions, is theemergence of a basis foritem-
construction requiring mentalmodels. Mental models are founded in theory:
and test items have become increasingly dependent on the verification of
cognitive models in whole or part. These have been sought, applied, and
clarified using dtfferent approaches to item generation owing very little to
each otherbut contributing essential parts of a newtheory of test
development.
Good illustrations ofhow the earliest adventures in itemgeneration
could influence later test development are immedtately evident in the
chapters written byBejar and Hornke. Both were concerned to generate
figtrral' items. Bejar's work in the 1980s (Bejar, 1986a, l986b, 1986c)was
based on two unique and seminal approaches. First, he stipulated that to
construct items a l o g d syntax, or grammar was essential. This made explicit
the need for a verifiable internal logic used by the computer to produce
items. Second, he applied an objective method (the Hough [1962]
transform) to produce an external, computer-drivenreferentforitem
dtfficulty based on figural complexity. With these two original moves, Bejar
providedan ear4definition of theprocess of itemgeneration that was an
indispensable key to its understandtng. He also based test development of
spatial orientation items derived from three-dtmensional representations on
Roger Shepard's chronometricmodels (Shepard & Metzler, 1971), using
that paradtgm as a foundation for m a l n g items progressively more dtfficult
to solve. He thus used two dtfferent methods for item construction and
subsequent verification-one a machine-driven algorithm and the other a
specific mental model based in time to complete an item (see also Bejar &
Introductionxvii
2 Primitive is used hcrc in the sensc of cxplanatory but incapable of further explanation. 11s a
type of unitary scmantic entity it prcscnts barricrs to furthcr decomposition and explanation.
xviiiIrvine
RUNNING IT U P T H E FLAGPOLE
TO SEE IF IT WOULD FLY
Thus far, the introduction has concerned itself with three of the horsemen
of the computer-based testing apocalypse that may yet sweep away paper-
and-pencil forms.They ride under thebanners of cognitive models,
psychometricrobustness, and a h n i s t r a t i v e convenience. Thefourth
horseman carries the flag of political defensibdlty. Howard Wainer’s
generous agreement to write his comments on presentations as a separate
chapter provides a sharp focus for political issues. When and how should
itemgeneration be used andwhat are its advantages over tradltional
methods? Suffice itto say that Wainer arguescogently for a particular
outcome of the internal debates of major test-producers on how to deliver
andconstruct tests profitably while technology is forcing changes upon
constructors and users alike. It is c h a l l e n p g readlng; but answers to many
of the questions raised by him are either implicit or explicit in the chapters
Introductionxxiii
THE DISCUSSIONS
REFERENCES
X.—PLAY-HOUSE MEMORANDA
(1813)
I once sat in the Pit of Drury-lane Theatre next to a blind man, who,
I afterwards learned, was a street musician, well known about
London. The play was Richard the Third, and it was curious to
observe the interest which he took in every successive scene, so far
more lively than could be perceived in any of the company around
him. At those pathetic interviews between the Queen and Duchess
of York, after the murder of the children, his eyes (or rather the
places where eyes should have been) gushed out tears in torrents,
and he sat intranced in attention, while every one about him was
tittering, partly at him, and partly at the grotesque figures and
wretched action of the women, who had been selected by
managerial taste to personate those royal mourners. Having no
drawback of sight to impair his sensibilities, he simply attended to
the scene, and received its unsophisticated impression. So much the
rather her celestial light shone inward. I was pleased with an
observation which he made, when I asked him how he liked Kemble,
who played Richard. I should have thought (said he) that that man
had been reading something out of a book, if I had not known that I
was in a play-house.
I was once amused in a different way by a knot of country people
who had come to see a play at that same Theatre. They seemed
perfectly inattentive to all the best performers for the first act or
two, though the piece was admirably played, but kept poring in the
play-bill, and were evidently watching for the appearance of one,
who was to be the source of supreme delight to them that night. At
length the expected actor arrived, who happened to be in possession
of a very insignificant part, not much above a mule [? mute]. I saw
their faint attempt at raising a clap on his appearance, and their
disappointment at not being seconded by the audience in general. I
saw them try to admire and to find out something very wonderful in
him, and wondering all the while at the moderate sensation he
produced. I saw their pleasure and their interest subside at last into
flat mortification, when the riddle was at once unfolded by my
recollecting that this performer bore the same name with an actor,
then in the acme of his celebrity, at Covent-Garden, but who lately
finished his theatrical and mortal career on the other side the
Atlantic. They had come to see Mr. C——, but had come to the
wrong house.
Is it a stale remark to say, that I have constantly found the interest
excited at a play-house to bear an exact inverse proportion to the
price paid for admission. Formerly, when my sight and hearing were
more perfect, and my purse a little less so, I was a frequenter of the
upper gallery in the old Theatres. The eager attention, the
breathless listening, the anxiety not to lose a word, the quick
anticipation of the significance of the scene (every sense kept as it
were upon a sharp look out), which are exhibited by the occupiers of
those higher and now almost out-of-sight regions (who, going
seldom to a play, cannot afford to lose any thing by inattention),
suffer some little diminution, as you descend to the lower or two-
shilling ranks; but still the joy is lively and unallayed, save [that] by
some little incursion of manners, the expression of it is expected to
abate somewhat of its natural liveliness. The oaken plaudits of the
trunkmaker would here be considered as going a little beyond the
line.—In the pit first begins that accursed critical faculty, which,
making a man the judge of his own pleasures, too often constitutes
him the executioner of his own and others! You may see the
jealousy of being unduly pleased, the suspicion of being taken in to
admire; in short, the vile critical spirit, creeping and diffusing itself,
and spreading from the wrinkled brows and cloudy eyes of the front
row sages and newspaper reporters (its proper residence), till it
infects and clouds over the thoughtless, vacant countenance, of
John Bull tradesmen, and clerks of counting-houses, who, but for
that approximation, would have been contented to have grinned
without rule, and to have been pleased without asking why. The
sitting next a critic is contagious. Still now and then, a genuine
spectator is to be found among them, a shopkeeper and his family,
whose honest titillations of mirth, and generous chucklings of
applause, cannot wait or be at leisure to take the cue from the sour
judging faces about them. Haply they never dreamed that there
were such animals in nature as critics or reviewers; even the idea of
an author may be a speculation they never entered into; but they
take the mirth they find as a pure effusion of the actor-folks, set
there on purpose to make them fun. I love the unenquiring gratitude
of such spectators. As for the Boxes, I never can understand what
brings the people there. I see such frigid indifference, such
unconcerned spectatorship, such impenetrability to pleasure or its
contrary, such being in the house and yet not of it, certainly they
come far nearer the nature of the Gods, upon the system of
Lucretius at least, than those honest, hearty, well-pleased,
unindifferent mortals above, who, from time immemorial, have had
that name, upon no other ground than situation, assigned them.
Take the play-house altogether, there is a less sum of enjoyment
than used to be. Formerly you might see something like the effect of
a novelty upon a citizen, his wife and daughters, in the Pit; their
curiosity upon every new face that entered upon the stage. The talk
of how they got in at the door, and how they were crowded upon
some former occasion, made a topic till the curtain drew up. People
go too often now-a-days to make their ingress or egress of
consequence. Children of seven years of age will talk as familiarly of
the performers, aye and as knowingly (according to the received
opinion) as grown persons; more than the grown persons in my
time. Oh when shall I forget first seeing a play, at the age of five or
six? It was Artaxerxes. Who played, or who sang in it, I know not.
Such low ideas as actors' names, or actors' merits, never entered my
head. The mystery of delight was not cut open and dissipated for me
by those who took me there. It was Artaxerxes and Arbaces and
Mandane that I saw, not Mr. Beard, or Mr. Leoni, or Mrs. Kennedy. It
was all enchantment and a dream. No such pleasure has since
visited me but in dreams. I was in Persia for the time, and the
burning idol of their devotion in the Temple almost converted me
into a worshipper. I was awe-struck, and believed those
significations to be something more than elemental fires. I was, with
Uriel, in the body of the sun.—What should I have gained by
knowing (as I should have done, had I been born thirty years later)
that that solar representation was a mere painted scene, that had
neither fire nor light in itself, and that the royal phantoms, which
passed in review before me, were but such common mortals as I
could see every day out of my father's window? We crush the faculty
of delight and wonder in children, by explaining every thing. We take
them to the source of the Nile, and shew them the scanty runnings,
instead of letting the beginnings of that seven fold stream remain in
impenetrable darkness, a mysterious question of wonderment and
delight to ages.
REVIEW OF THE EXCURSION; A
POEM
By WILLIAM WORDSWORTH. London. 4to. pp. 447
(1814)
The volume before us, as we learn from the Preface, is "a detached
portion of an unfinished poem, containing views of man, nature, and
society;" to be called the Recluse, as having for its principal subject
the "sensations and opinions of a poet living in retirement;" and to
be preceded by a "record in verse of the origin and progress of the
author's own powers, with reference to the fitness which they may
be supposed to have conferred for the task." To the completion of
this plan we look forward with a confidence which the execution of
the finished part is well calculated to inspire.—Meanwhile, in what is
before us there is ample matter for entertainment: for the
"Excursion" is not a branch (as might have been suspected)
prematurely plucked from the parent tree to gratify an overhasty
appetite for applause; but is, in itself, a complete and legitimate
production.
It opens with the meeting of the poet with an aged man whom he
had known from his school days; in plain words, a Scottish pedlar; a
man who, though of low origin, had received good learning and
impressions of the strictest piety from his stepfather, a minister and
village schoolmaster. Among the hills of Athol, the child is described
to have become familiar with the appearances of nature in his
occupation as a feeder of sheep; and from her silent influences to
have derived a character, meditative, tender, and poetical. With an
imagination and feelings thus nourished—his intellect not unaided by
books, but those, few, and chiefly of a religious cast—the necessity
of seeking a maintenance in riper years, had induced him to make
choice of a profession, the appellation for which has been gradually
declining into contempt, but which formerly designated a class of
men, who, journeying in country places, when roads presented less
facilities for travelling, and the intercourse between towns and
villages was unfrequent and hazardous, became a sort of link of
neighbourhood to distant habitations; resembling, in some small
measure, in the effects of their periodical returns, the caravan which
Thomson so feelingly describes as blessing the cheerless Siberian in
its annual visitation, with "news of human kind."
In the solitude incident to this rambling life, power had been given
him to keep alive that devotedness to nature which he had imbibed
in his childhood, together with the opportunity of gaining such
notices of persons and things from his intercourse with society, as
qualified him to become a "teacher of moral wisdom." With this
man, then, in a hale old age, released from the burthen of his
occupation, yet retaining much of its active habits, the poet meets,
and is by him introduced to a second character—a sceptic—one who
had been partially roused from an overwhelming desolation, brought
upon him by the loss of wife and children, by the powerful
incitement of hope which the French Revolution in its
commencement put forth, but who, disgusted with the failure of all
its promises, had fallen back into a laxity of faith and conduct which
induced at length a total despondence as to the dignity and final
destination of his species. In the language of the poet, he
——broke faith with those whom he had laid
In earth's dark chambers,
"I have seen," the poet says, and the illustration is an happy one:
——I have seen
A curious Child [who dwelt upon a tract
Of inland ground], applying to his ear
The convolutions of a smooth-lipp'd Shell;
To which, in silence hushed, his very soul
Listened intensely, and his countenance soon
Brightened with joy; for murmurings from within
Were heard,—sonorous cadences! whereby,
To his belief, the Monitor expressed
Mysterious union with its native Sea.
Even such a Shell the Universe itself
Is to the ear of Faith; and [there are times,
I doubt not, when to you it] doth impart
Authentic tidings of invisible things;
Of ebb and flow, and ever during power;
And central peace subsisting at the heart
Of endless agitation.—p. 191.
Sometimes this harmony is imaged to us by an echo; and in one
instance, it is with such transcendant beauty set forth by a shadow
and its corresponding substance, that it would be a sin to cheat our
readers at once of so happy an illustration of the poet's system, and
so fair a proof of his descriptive powers.
Thus having reached a bridge, that overarched
The hasty rivulet where it lay becalmed
In a deep pool, by happy chance we saw
A two-fold Image; on a grassy bank
A snow-white Ram, and in the crystal flood
Another and the same! Most beautiful,
On the green turf, with his imperial front
Shaggy and bold, and wreathed horns superb,
The breathing Creature stood; as beautiful,
Beneath him, shewed his shadowy Counterpart.
Each had his glowing mountains, each his sky,
And each seemed centre of his own fair world:
Antipodes unconscious of each other,
Yet, in partition, with their several spheres,
Blended in perfect stillness, to our sight!—p. 407.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookname.com