Professional Documents
Culture Documents
Preface
The operational definitions of the PES performance indicators Q6 and Q7 have been agreed upon by the Participating Partners at their second meeting in Vienna in March 2006. The paper presents the conclusions the team of consultants has drawn from the discussion during the meeting. This draft paper is subject to revisions. These revisions might turn out to be substantial.
For the team of consultants: Prof. Michael WagnerPinter Vienna, 27 July 2006
Content
1 2 indicators Q6 and Q7 3 Satisfaction 4 Are the data indicators available? already How part are of they the defined? PES information system? 7 6
1 Satisfaction Survey Variations Standard Operational is in survey of Satisfaction indicators Q6 and Q7 definition job seekers indispensable survey scenario design ofand
employers indicator
The two performance indicators Q6 and Q7 are from an operational point of view quite the same. Both indicators aim at capturing the satisfaction with the overall service the PES provides to customers. Q6 is directed towards people (as potential job seekers), whereas Q7 is directed towards employers. The empirical information for Q6 and Q7 has to be survey based. Thus, the PES either has already established such a survey (and is therefore in a position to retrieve the information); or it has not (yet) established the survey and can therefore not retrieve the information. Of course, there might be a substantial variation in the way the survey is carried out (methods of sampling, frequency of the survey, type of questions asked), but this will not be dealt with at this stage of the project. There are, however, two issues that have to be addressed in the context of operational definitions. Let us start with the following standard scenario for a satisfaction survey. This standard survey includes the question How are you satisfied with the overall services the Public Employment Service has delivered? structures the answers to the questions into the items very satisfied satisfied unsatisfied very unsatisfied Within the scenario of the standard survey the operational definitions of indicators Q6 and Q7 are straightforward: (1) Count the number of all answers
Best case, Deviations Overall School grading worst are case likely satisfaction system
(2) choosing the items very satisfied and satisfied. (3) Divide this number (4) by the number of all answers returned to this question. The indicators will assume the value 1 if all customers are satisfied; it will assume the value 0 if none is satisfied. It is likely that the practice of several PES among the Participating Partners will deviate from the standard survey scenario. In particular, two deviations might occur: Firstly, all questions asked refer to specific services delivered. Secondly, the answers are structured along a grading which reflects the grading used in the respective national school system. Regarding the first deviation, the overall satisfaction can be derived as a weighted average of the answers to the specific service questions. The weights used in this procedure should be documented when the information is forwarded. Regarding the second deviation, there is a simple solution. Just partition the set of grades in a top half and a bottom half if the number of grades is even; if the number of grades is odd, then delete the neutral grade between the top and the bottom set. Once this partition has been carried out, it is possible to use the operational definition given; it is only necessary to substitute the line (2) by choosing grades from the top half.
2 3 Customer satisfaction (employers) (job seekers) Are the data indicators available? already How part are of they the defined? PES information system?
Q6: Achieving customer satisfaction (job seekers) Is this indicator already part of your management information system? Is there an indicator already part of your management information system that comes close to Q6 with regard to data provision
yes
no
Q7: Achieving customer satisfaction (employers) Is this indicator already part of your management information system? Is there an indicator already part of your management information system that comes close to Q7 with regard to data provision
yes
no
ListPapers White Blue Red Yellow Green Papers of Papers Papers papers of the Working Group
All papers of the Working Group will be distributed by the group of consultants to each of the Participating Partner: There are several series of papers. Some of them can be used for general circulation, some of them are for internal use only; none of the papers are strictly confidential. White Papers deal with general topics and cover the topics of the final report of the Working Group (unrestricted circulation) Blue Papers list performance indicators and discuss their strong and weak points (unrestricted circulation) Red Papers report on harmonized data forwarded by the Members of the Working Group (circulation only with permission of the Members who provided data) Yellow Papers contain only information which serves to organize the work in progress (of no use to outsiders of the Working Group) Green Papers cover issues related to the choice of good practices and their presentations (unrestricted circulation)
Reference WP 01 BP 01 BP 04 BP 06 BP 10
Title PES-Performance, Indicators, Good Practice Supporting Flowcharts Measuring effectiveness and operational efficiency of the Finnish PES A list of eight performance indicators (Second draft) Measurement Procedures: The specific approaches taken by the Participating Partners in calculating the performance indicators and the underlying populations (First Draft) Capturing Context: A radar chart approach (First draft) Operational definitions of performance indicators Q1Q3 (Fourth draft) Operational definitions of performance indicators Q4 and Q5 (Second draft) Operational definitions of performance indicators Q6 and Q7 (First Draft) Operational definition of performance indicator Q8 (First Draft) Public Employment Services in Europe: a large spectrum of missions Performance Indicators: A first survey on data availability and ranking PES Background and Content Information and Indicators in the year 2005 (Second draft) A tight schedule (Third draft) Transition from unemployment to employment: Indicator 1 Transition from unemployment to employment before unemployment lasts long: Indicator 2 Transition from measures to employment: Indicator 3 Looking for Good Practices (First Draft)
8
BP 11 BP 12 BP 13
BP 14
27/07/06
BP 15
27/07/06
Yellow Papers
YP 01 YP 02
11/01/06 12/01/06
YP 06
12/05/06
YP 07 Red Papers RP 01 RP 02
RP 03 Green Papers GP 01
30/05/06 01/06/06
List of Papers (previous drafts) Type of paper Blue Papers Reference BP 02 Title Transition from Unemployment to Employment (First draft) Transition from Unemployment to Employment (Second draft) A preliminary list of eight performance indicators (First draft) Operational definitions of performance indicators Q1Q3 (Third draft) Operational definitions of performance indicators Q4 and Q5 Operational definitions of performance indicators Q6Q8 A tight schedule (First draft) PES Background and Content Information and Indicators (First draft) A tight schedule (Second draft)
BP 03
28/02/06
BP 05
06/03/06
BP 07 BP 08
03/04/06 03/04/06
BP 09
03/04/06
Yellow Papers
YP 03 YP 04 YP 05
Eigentmer und Verleger: Synthesis Forschung Gesellschaft m.b.H. Gonzagagasse 15/3 1010 Wien Fr den Inhalt verantwortlich: a.o. Univ.-Prof. Dr. Michael Wagner-Pinter Synthesis Forschung Wien 2006