Address Concerns About Knowledge Tests
The National Association of Flight Instructors (NAFI) joined FAA
staff, other industry representatives, and university experts in
Oklahoma City recently to review how the agency creates airman
knowledge-test question banks-a process that corrected problems in
the recently revised Fundamentals of Instruction test.
Education experts from university aviation education programs,
including Embry-Riddle Aeronautical University, Liberty
University, the Professional Aviation Board of Certification,
University of North Dakota, the University of Oklahoma, and Western
Michigan University, participated in the meeting at the request of
NAFI and AOPA, as a follow-up to discussions held several weeks
ago. The group specifically addressed the Fundamentals of
Instruction test; rates of failure for that test have increased
significantly since the bank of questions used to create the test
was updated on February 14, 2011, Blair said.
"We took a closer look at the specific questions that were added
to the test bank and evaluated the test question structure, answer
options, and reference for each question," said NAFI Executive
Director Jason Blair. "Six questions were identified for removal
from the test bank, and a few others that were identified for
revision. The FAA immediately addressed those questions, and it
will review the results of applicants who failed the test on the
basis of these questions."
Those applicants will be contacted directly by an FAA staff
member, and a revised knowledge test score will be issued, Blair
said.
In spite of that correction, the group noted that many of the
new questions are directly drawn from the most recent update of the
Aviation Instructors Handbook. Some of these questions continue to
result in poor application performance, but they're well written
and based solidly on the content in the handbook, Blair said. That
reinforces that it should be used as the primary text for
applicants preparing for this particular knowledge test. Review of
results from other knowledge tests suggest that both pass rates and
overall scores haven't changed enough to warrant further review of
other test banks, Blair said.
"Both the FAA staff and industry representatives strongly
support an approach where students study for airman knowledge tests
by developing their knowledge of the material, not by studying
specific test questions," Blair said. "We're working together to
create an understanding of how to best prepare applicants for tests
while improving overall knowledge and safety in our aviation
system."
In addition to the test-question review, university experts also
addressed test question development and validation, and they
provided feedback on some methods of test development that are
generally accepted in the educational community. The FAA recognizes
that it can learn from the education community, Blair said. As a
result, it will continue this discussion with an industry steering
committee it expects to form in the near future, which will be
comprised of educational and industry representatives who can
provide feedback on test development and validation.
"NAFI strongly supports the continued effort of the FAA to
ensure a testing process that accurately evaluates applicants'
understanding of relevant material," Blair said. "We'll continue to
offer our assistance and resources to support this effort, and
disseminate information about future changes to the
flight-instructor community."
NTSB Final Report: Douglas A-4K
See All