Item writing

guidelines
Manual

Version 2 April 2014

Table of Contents
Introduction .............................................................................................................................................................. 1
General guidelines.................................................................................................................................................... 2
Certification levels ................................................................................................................................................. 2
Target audiences for SAP exams ......................................................................................................................... 3
Relevance ............................................................................................................................................................ 4
Conciseness ......................................................................................................................................................... 6
Language ............................................................................................................................................................. 6
Question types.......................................................................................................................................................... 7
Multiple choice ...................................................................................................................................................... 9
Multiple response................................................................................................................................................ 10
Item writing process ............................................................................................................................................... 11
Item writing rules .................................................................................................................................................... 13
Overview ............................................................................................................................................................ 14
Rules .................................................................................................................................................................. 16
Examples ................................................................................................................................................................ 19
Pronouns ............................................................................................................................................................ 19
Single or multiple sentence stems....................................................................................................................... 21
Answers .............................................................................................................................................................. 23
Terminology ............................................................................................................................................................ 24

Exercise .................................................................................................................................................................. 25

Introduction
Writing test items for a certification exam is a creative process. Item writing rules help in this process but they are
not a replacement for your experience and creativity. We cannot guarantee that you will always write good items
if you follow the rules. However, we can guarantee that you will not write good items if you ignore the rules.

The plan
We start by describing the general guidelines for item writing. We explain the basic item types that you can use to
formulate your questions (multiple choice and multiple response). We continue with a description of the item
writing process and present the core item writing rules. We conclude with some examples of good and bad items.

Why should you be interested in following the rules?

Quite simply: if you follow the rules, your items have a good chance of being accepted. The more items
we can accept, the closer you get to Bronze, Silver, or Gold status. (See the SAP Community Network for
more details on the rewards.)

A few words of caution

There is nothing creative in copying test questions from existing material. If you remember a test item from a
certification exam you passed (or not), then do not submit this item. If you remember a test item from training
material, then do not submit it. If you plan on submitting a publicly available (sample) question, dont do it. We
already know these questions. We will not accept them. You will not receive credit for them and will just be
wasting your time.

SAP Item writing guidelines

General guidelines
You must have passed at least one SAP certification exam. This means you are already familiar with SAP exam
questions. We will try to guide you through the process of writing them.

Certification levels
SAP currently offers three levels of certifications: Associate, Professional and Specialist. SAP certification exams
are typically based on job and task analyses that help to distill the important knowledge and competencies an
exam will assess.

Associate
Associate exams test broad, fundamental knowledge and competencies in an SAP solution or methodology.
Associate exam candidates may have project experience in the SAP solution for which they want to be certified.
Normally this is not a requirement unless stated in the exam description.

Professional
Professional exams usually target consultants with four or more years of project experience in the related SAP
solution.

Specialist
Specialist exams target consultants who have gained an Associate certification in a particular solution and
chosen to validate their specialist knowledge in an associated area.

Consequence for your contribution

If you write test questions for an Associate or Specialist exam, you should be familiar with the training material so
that you have an idea of how SAP envisions the respective role.

If you write test questions for a Professional exam you should have worked for at least four years in the
respective SAP solution domain.

SAP Item writing guidelines

Target audiences for SAP exams

We want to test whether candidates know enough in the domain of the certification exam. Knowing enough is,
of course, hard to define but normally you know it when you see (or dont see) it. Someone knows enough in the
exam domain if she or he can contribute positively to a project. Someone knows enough if she or he does not
cause damage due to lack of knowledge in the exam domain.

We are not trying to identify the top performers. If we can identify the ones who know enough, the ones who are
sufficiently qualified, we have achieved our goal.

This means that you should write questions that you would ask someone to find out whether she or he knows
enough. Your questions work like a filter.
If the questions you ask are too easy, they will not act as a filter because everyone will answer them correctly. If
your questions are too difficult, only a few will pass your filter.
Therefore, write the questions that you think perhaps half of the candidates would answer correctly.

SAP Item writing guidelines

Relevance
You should write test questions that matter. They should require the exam candidate to think. They should not just
require the candidate to retrieve trivia from her or his memory.

If you want to write a question about something, think first about what the lack of knowledge could do to an
implementation project:

Would the project be delayed or would customer satisfaction be reduced? If so, the chances are that you
are on to something that is relevant.
Does the subject of your question come up in almost every project? If so, the chances are that you
are on to something that is frequent.

Anything that is somewhere else might be important but rare, so it cannot be considered typical.

SAP Item writing guidelines

A few words of caution

Be aware of an availability bias:
Just because you experienced something in your last project does not mean it is important in most other
projects.
Just because you saved the day with a particular insight does not make this a good subject for a test
question. Kudos to you but it may be too exotic.

Be aware of opinion: exams force a candidate to make clear choices by selecting one, two, or three correct
answers. There is no place for discussion. If something is a matter of opinion, the chances are that the correct
answer would start with It depends We cannot fail someone just because she or he has a different opinion.
Would you like to fail an exam because you have a different opinion?

SAP Item writing guidelines

Conciseness
Certification exams do not instruct. They test whether someone knows enough or is competent enough. If she or
he did not learn something she or he was supposed to have learned, it is too late. Therefore, do not explain
general mechanisms, concepts, terms, etc. If you want to use these terms in a test question, then you expect that
the exam candidate already knows this.

You have taken certification exams. Every minute mattered. Dont waste your time writing lengthy questions.
Dont waste an examinees time by forcing her or him to read a bunch of words that convey little or no
information.

More words mean more opportunities for confusion. If you use as few words as possible, you can reduce the
chances of causing confusion. Write your questions to the point. We dont window dress questions.

Language
Your submissions must be in English. Preferably, the questions should also be grammatically correct.

SAP Item writing guidelines

Question types
SAP certification exams currently use selected response questions: exam candidates select one, two, or three
answers from a set of answers.
The question format of selected responses helps SAP ensure fairness for native and non-native speakers of
English alike.

We do not use questions that have only two answers. This means that you cannot ask questions like
Can you transfer data models? or Are data models transferable?.

Multiple choice (MC)

The question is presented with four answers of which one is correct. The candidate chooses the correct answer by
selecting a radio button that is displayed in front of an answer.

Multiple response (MR)

This question type has either four or five answers.

Dichotomous scoring
Each question yields one point if the correct answers and only the correct answers have been selected. There is
no partial credit for selecting a subset of correct answers.

SAP Item writing guidelines

Parts of a question
A question consists of two parts:
Stem

Answers

At a minimum the stem consists of one sentence: the question. The stem can also be a composite: a scenario or
mini story and the final question.

A scenario describes a specific business situation, for example events, data, actions taken, actions
planned, and errors. It does NOT explain concepts or terms or anything the candidate is supposed to
know at the time of the exam.

SAP Item writing guidelines

Multiple choice
In SAP exams multiple choice (MC) questions always contain four answers. Of these one, and only one, is
correct. The candidate can only select one answer.

When you submit a MC question to the SAP Crowdsourced Certification Questions Initiative the first answer must
always be the correct one. This is simply to avoid confusion. In an SAP certification exam, answers are always
shuffled.

The shuffling of answers has one major consequence: you cannot use anything like All of the
above, None of the below. Since answers are shuffled, an answer will be presented at any given
position.

SAP Item writing guidelines

Multiple response
In SAP exams, this question type comes in two flavors:

MR1: two correct answers, two incorrect answers, a total of four answers.

MR2: three correct answers, two incorrect answers, a total of five answers.

We always tell candidates how many answers they have to select.

As already discussed under dichotomous scoring, candidates receive one point if they select the correct answers.
We do not award partial points.

Multiple response (MR1): four answers, two correct

Choose this question type if there are two parts to a correct answer, for example two ways of accomplishing
something, two settings that must be made, two conclusions that can be drawn.

Multiple response (MR2): five answers, three correct

Choose this question type if there are three parts to a correct answer.

SAP Item writing guidelines

10

Item writing process

We recommend that you adhere to the following process.

Plan

Currently you have to choose the item type first. If you want to change your mind later, simply go back
to the web browser.

After you select the item type, you can select a topic from a list, for example

SAP Item writing guidelines

11

1. Pick one topic you are very familiar with, for example, Administering Data Models.
2. Think about a relevant task that you frequently perform in this area.
3. Write this task down, for example, Review the history of activated models.
4. Determine the knowledge or skill required to perform this task successfully.

5. Pick a result of your analysis and write a question about it.

Write
Start with the stem. Get a very clear idea about what you want to know from the candidate. The clearer the stem,
the clearer the answers will be. If you are fuzzy about what you want to know, chances are that your answers will
be fuzzy too. We try to avoid fuzzy questions!

After you have written the stem:

1. Write the correct answer or the correct answers, depending on the question type that you chose.
2. Enter the source information for the correct answer(s).
3. Write the incorrect answers.

4. Verify everything.

Submit
You are now good to go. After you submit the question, you will have a chance to review everything. Once you
are satisfied, submit the question. You can see your submissions but you cannot change them.

SAP Item writing guidelines

12

Item writing rules

We ask you to follow eight important rules. If you follow these rules all of them the odds are pretty good that
your submission will be accepted.

1. Specificity
2. Content and level
3. Fairness
4. Brevity
5. Proper English
6. No negatives
7. Parallel answers

8. Terminology

Before we continue and describe the rules in detail, let us outline some general best practices that will increase
your chances of submitting a question that we will accept.

SAP Item writing guidelines

13

Overview

Stem
No matter how many sentences you write for the stem (and it should not be too many), the last one must always
be the question.
While this may sound obvious, we receive quite a few submissions that do not contain a question. The least we
expect is a complete sentence that ends in a question mark. The word order of a question sentence is of course
a bit different from a statement. A sentence that ends in a colon or with continuation points () is simply not a
valid stem. This will almost always lead to your submission being rejected.

Answers
Every single answer must be a grammatically adequate answer to the question. Every single answer must be
self-contained and not refer to another answer in any way.

Correct answers
A correct answer is demonstrably and clearly correct. An answer must be correct beyond reasonable doubt. It is
not sufficient for an answer to be almost correct, for example, because all other answers are false.

Incorrect answers
Incorrect answers are clearly false in a non-trivial sense. In selected response questions, the incorrect answers
serve as distractors for a candidate who is not competent (enough). Even incorrect answers have to be fair.
Therefore, we do not allow answers as distractors that are too true to be true. Incorrect answers can be hard,
but they must not be sneaky or malicious. From a practical point of view, you must not invent, for example,
system options, method names, transaction names, and so forth.

SAP Item writing guidelines

14

Source information
We use this information to easily verify the correctness of the answer(s). Be as specific as possible when you
provide this information. Only use sources that are, or can be, available to the general public.

SAP Item writing guidelines

15

Rules
Rule 1: Specificity
The question is non-trivial and specific to the exam and usually starts with one of the six question pronouns
described below. The question stem avoids anything like, Which of the following statements are true?
First, a question stem like this could be used in any exam. There is nothing specific about it.
Second, the question stem is sometimes called a stimulus. A stimulus is supposed to trigger something. In the
case of certification questions, the stimulus is supposed to evoke a specific response, for example, a definition or a
rule, or a procedure to accomplish a task or to figure out how to solve a problem.
A stimulus should make the examinee think about a particular topic. Therefore, a question stem like Which of the
following statements is true? does not evoke anything. The examinee must read the answers first to get an idea of
what we wanted to ask. In our experience, the answers in this case are a hodgepodge of things that lack clarity.
We think that this is unfair to examinees. We are asking examinees to take a high-stakes exam. The least we can
do is to confront the examinees with well-thought-out questions; difficult maybe, but to the point and fair.

Never try to submit questions like Which of the following statements is true? or Which of the
following statements are not correct? or Which of the following statements regarding authorizations
is false? We cannot, and we do not, want to list all possible ways of phrasing this kind of poor
excuse for a question. You submit it, we will reject it!

Rule 2: Content and Level

The question fits the topic, is representative of the subject matter, and is adequate for a candidate of average
competence for a particular exam.

Dont write trivial questions. Dont write questions only a handful experts on the planet could perhaps
answer.

SAP Item writing guidelines

16

Rule 3: Fairness
The question is fair, clear, suitable for an international audience, and gender-neutral. The question avoids jokes,
idioms, and colloquialisms.

Fair: Dont write honey traps.

Clear: Dont cloud what you want to say by using many adjectives, etc. Write plain and simple
questions!
International audience: SAP exams are taken in over 50 countries. You cannot take anything for
granted.
Gender-neutral: No he, no she, no John, no Jane.
No idioms: Not even once in a blue moon can you write a question that asks for an eleventh hour
change
Colloquialism: Be precise in the terminology you use.

Rule 4: Brevity
The question is written concisely with as few words as possible. Most words are put in the stem and not in the
answers. Do not explain anything you expect the average competent candidate to know.

Rule 5: Proper English

Write in Standard English, using the active voice, and present tense. The stem always ends in a question mark (?).

Rule 6: No Negatives
Negatives can be confusing or are easily overlooked. Avoid negatives such as not in any combination, for
example, does not, cannot, could not, should not, or must not.
Examples:
Positive and conforming to the rules: Which method can you use to create an index?
Negative and violating the rules: Which method can you NOT use to create an index?

Negative and violating the rules: Which method should you avoid when you create an index?

There is one exception: When you are writing a troubleshooting question, you could describe a
scenario or an action that fails and ask about the reason(s), you could describe a failed action and
ask how to correct this, or you could describe a failed operation and ask how the failure could have
been prevented. In these cases you might see the need to use one negative. If you use a negative in
the stem you cannot use negatives in answers.

SAP Item writing guidelines

17

Rule 7: Parallel answers

Answers should not give away a clue to the question. Therefore ensure that:
All answers match the question.
The wording of the answers does not hint at their correctness or incorrectness.
All answers are of roughly equal length.
All answers are phrased in a parallel way.
All answers are meaningful and do not refer to other answers.
All answers are plausible and nothing is made up.
Answers such as All or None of the above/below are never used.

Try keeping the answers as short as possible. If all the answers have words in common try to put these into
the stem.

Rule 8: Terminology
The terminology (SAP and otherwise) must be correct. Do not make up anything, for example, do not invent
method names, transaction names, system settings, and so forth. The capitalization of terms must follow the
official guidelines. When you refer to transactions, use the descriptive name of the transaction and put the
transaction code in parentheses, for example, the Workbench Organizer (SE09) transaction. You can find
approved names of all SAP products at
SAP Branding Tools Naming.
You can use abbreviations after you introduce the long form first. Put the abbreviation in parentheses after the
long form. Then you can continue to use the abbreviation. Typically, you want to do this in the question stem.
Since answers are presented in random order, it is usually neither necessary nor useful to introduce
abbreviations in answers.

SAP Item writing guidelines

18

Examples
The following examples provide you with some ideas on how to write questions.

Pronouns
Most of your questions will start with one the following question pronouns:
1. What
2. When
3. Where
4. Which
5. Why

6. How

What questions
What can you use to update data source settings on the system database?
What information must you collect before you can run an XYZ report?

What data is included in the sales report?

SAP Item writing guidelines

19

When questions
When would you install the Advanced Adapter Engine decentrally?
When are xyz tables created?

Where questions
Where is this change reflected?
Where do you set the xyz parameter?

Which questions
Which of the following statuses should a work item have if you want to archive it?
Which of the following must exist before an agent can create an automatic billing document?
Which indicators must you define to enable a performance trace?

You might say But didnt they forbid the use of Which of the following? We do not allow things
like Which of the following statements is true? You can use Which of the following if you need to
limit the choices you list in the answers. For example, if you want to ask for system options but
cannot list them all, use Which of the following system options can you activate to That way a
candidate knows that the list provided is not exhaustive.

Why questions
Why would you use the test tool of the Business Object Repository (SWO1)?
Why would you build an SAP BusinessObjects BI universe on top of SAP HANA using the Information Design
Tool (IDT)?

How questions
How do you implement?
How does the system behave if you have an Idle Connection Timeout setting?
How can you transport content?

Keep in mind: All answers must be grammatically correct answers to the question pronoun
you choose.

SAP Item writing guidelines

20

Single or multiple sentence stems

Single sentence stems
This is the shortest way of asking a question. You start with one of the pronouns listed in the previous section and
complete the sentence following the rules.

What is the valid sequence for processing scheduled meter readings?

When would you consider setting up an SAP data archiving project?

You would probably use single-sentence questions if you are interested in finding out whether a candidate has
understood a concept, a procedure, or a process. Typically, short questions test memorization of things because
it is difficult to describe a situation and ask a question about this in one (comprehensible) sentence. Although it is
always possible to write one long and winding sentence, this would go against the rules described above. Long
and winding sentences are more difficult to understand for linguistic reasons alone. Write in terse prose. Split up
what you want to say into logical units rather than trying to fit them into one monstrous construction. Keep in mind
that a candidate has about two minutes to read, comprehend, and answer an exam question.

Sometimes you may want to stress the context of a question. In this case you could put the context first. For
example In IT Service Management, how can you.

Multiple sentence stems

You use multiple sentences to describe a specific situation before you ask the question. First, set up a scenario
a typical and specific business situation that includes events, data, actions taken, actions planned, errors, and so
on. Second, ask the question (see single sentence question above). Common situations would be:

Describing a specific (business) situation, goal, or error

Describing an action or actions

Describing (business) requirements

Examples
1. Description of a specific [business] situation, goal, or error
2. One 6W question

SAP Item writing guidelines

21

Billing fails to produce a billing document due to a missing move-in meter reading. What can you configure to
notify the back office of this failure?

A tariff requires that for the same contract account, the consumption from the water meter is used to bill the
water consumption charge and the waste water consumption charge. How can the water meter be installed to
meet this tariff requirement?

1. Description of an action or actions

2. One 6W question about consequences or actions
You change the title of a Crystal Reports object using the Central Management Console. Where is this
change reflected?

Your customer has created a custom program, Z_ CUST_ PRG, that updates custom table Z_ CUST_ TBL. The
Firefighter ID is used to execute the Z_ CUST_ PRG program during a Firefighter session. How do you ensure
that the Firefighter log includes changes made to the Z_ CUST_ TBL table?

1. Description of an action or actions

2. Description of unexpected consequences

3. One 6W question about problem resolution, analysis, or causes

A user has started an instance of a workflow. The first step of the workflow is an approval step that should have
created a work item for the manager. However, the manager did not receive the work item. What should you do
first to analyze the cause of this issue?

You create a workflow that starts when the event EQUI. Created is raised (a new equipment is created). The first
step is to display this equipment using the EQUI.Display method. However, this method is not receiving the
equipment object. What binding would you add to resolve this problem?

SAP Item writing guidelines

22

Answers
Answers are always presented in random order, that is, you cannot rely on a specific answer to be on top or at
the bottom. Hence, pseudo answers such as All of the above, All of the below, None of the above, and so
forth are meaningless and therefore not allowed. An answer such as All of the above would normally violate the
rule that all answers must be linguistically correct answers to the question asked.

According to rule 6 regarding parallel answers, all answers must match the question from a content or language
point of view. All answers should be of roughly equal length, phrased in a parallel way, be meaningful, and stand
for themselves.

SAP Item writing guidelines

23

Terminology

Term

Meaning

Exam question

An exam question comprises the elements:

Question stem
Correct answer(s)
Incorrect answers
Source information for correct answers.

Correct answer

The answer(s) a candidate must select to score a point.

Dichotomous scoring

A candidate scores a point if the candidate selects the required number of correct
answers. For multiple choice questions, this is 1 answer. For multiple response
questions, this is either two or three answers. Failure to select the correct number of
answers results in zero points. No partial credits are awarded.

Incorrect answer

A false answer, also called a distractor

Item

Short for test item, the same as an exam question

Multiple choice
question (MC)

A question with:
One correct answer

Three incorrect answers

All answers are displayed with one radio box; thus it is only possible to select
one answer.
Multiple response
question type 1 (MR1)

A question with:
Two correct answers
Two incorrect answers
The candidate must select the two correct answers to score one point.

Multiple response
question type 2 (MR2)

A question with:
Three correct answers
Two incorrect answers
The candidate must select the three correct answers to score one point.

Question stem

The question, which consists of:

Zero or more sentences that describe a scenario
Exactly one grammatical question
The minimum length is 50 characters.

Validation or source
information

Precise and accurate information that must be supplied to enable SAP to validate the
correctness of an answer.

SAP Item writing guidelines

24

Exercise

No.

Rule 1:
Specificity

Example

Rule 2:
Content and
level

Rule 3:
Fairness

Rule 4:
Brevity

Rule 5:
Proper
English

Rule 6:
No
negatives

Rule 7:
Parallel
answers

Rule 8:
Terminology

Afaria installation pack

doesnt consists of

Choose main
functionalities of Data
Orchestration Engine.

What will you use to
fulfill this requirement?

What can be used to
filter a table on multiple
selection criteria?

Which of the following

statements regarding
CUA is correct?

SAP Item writing guidelines

25

No.

Rule 1:
Specificity

Example

Rule 2:
Content and
level

Rule 3:
Fairness

Rule 4:
Brevity

Rule 5:
Proper
English

Rule 6:
No
negatives

Rule 7:
Parallel
answers

Rule 8:
Terminology

Connections allow
Unwired Server to
communicate with
data sources. At what
level in the
environment are
connections managed
and maintained?

Certain elements of a
system landscape are
crucial for the
availability of the
system. These
elements are
considered Single
Points of Failure since
if they stop working,
the entire system will
be unavailable.

What elements are

considered Single
Points of Failure?
8

Which of the following

statements are not
true?

SAP Item writing guidelines

26

No.
9

Rule 1:
Specificity

Example

Question: How can

you identify the
pricing function within
a trace file?

Rule 2:
Content and
level

Rule 3:
Fairness

Rule 4:
Brevity

Rule 5:
Proper
English

Rule 6:
No
negatives

Rule 7:
Parallel
answers

Rule 8:
Terminology

Answers:
You cannot identify
the function in a trace
file.
10

Question: What
happens when the
config file is
accidentally deleted?
Answers: All nodes
in the multiplex will
shut down. All
writers in the
multiplex will shut
down. Another
node must be
manually promoted to

SAP Item writing guidelines

27

No.

11

Rule 1:
Specificity

Example

Question: Can
additional indexes
be useful for
improving
performance?

Rule 2:
Content and
level

Rule 3:
Fairness

Rule 4:
Brevity

Rule 5:
Proper
English

Rule 6:
No
negatives

Rule 7:
Parallel
answers

Rule 8:
Terminology

Answers: Yes,
because a database
will use all
information. No,
because one index is
sufficient. Yes,
because indexes can
be combined by the
optimizer. No,
because the optimizer
cannot combine
multiple indexes.

SAP Item writing guidelines

28