WiBe-TEAM has been partner in the eGovernment monitor network ‘eGovMoNet’, promoted by the European commission in its Competitiveness & Innovation Framework Programme (CIP). eGovMoNet shared experiences with numerous European public organizations and providers involved in e-Government, and investigated the impact of e-Government services.
The implementation of eGovernment solutions is expected to bring a number of advantages including efficiency, increased user satisfaction, and reduction of administrative burden. However, this cannot be taken for granted any. Well informed strategic decisions on investments in eGovernment developments will need measurements of impact and user satisfaction.
The eGovMoNet therefore proposes to raise the awareness and stimulate uptake of e-Government impact and user satisfaction measurements and to contribute to the harmonisation and quality of measurement frameworks. National concepts (which very much vary) are analyzed and compared. After the kick off workshop May 2008 in Geneva more results have been presented during the following workshops.
eGovMoNet: kick-off meeting at the United Nations in Geneva
WiBe-TEAM PR recently joined eGovMoNet a new European thematic network aimed at measuring the user satisfaction of e-Government solutions. A series of eGovMoNet events brings together some of the most authoritative experts on measurement strategies, focusing their skills on the specific challenge of measuring in a consistent yet easily applicable way the levels of impact and satisfaction relative to a great diversity of eGovernment services and user catchments areas.
Meeting report Geneva, 29-30 May 2008
Among the purposes of the eGovMoNet kick-off meeting, apart from setting a foundation stone for future events, and letting people make acquaintance with each other, we find the following:
- Align the mutual expectations to the project and the collaboration.
- Establish a common understanding of the plans, administrative procedures and prospects of project.
- Start collection of information as input for the project work and deliverables.
- Clarify some initial unresolved issues.
- Get to know some of the core measurement methods currently in use.
- Introduce the first thoughts on how to describe the methods to compare them.
In a very collaborative environment, Professor Mikael H. Snaprud (University Agder), the coordinator of eGovMoNet, acted as facilitator and ‘master of the house’ for the first meeting. The complexity of the challenges facing the eGovMoNet network, with tasks including harmonization of strategies and criteria, and the fact that there are still unexplored aspects to deal with, can’t be overestimated.
Despite this pioneering side of eGovMoNet, the talks of this kick-off meeting were very practical and operational in nature, and included among others:
- Pascal Collotte proposed fostering of thematic networks as an useful tool to exploit the potential of EU Competitiveness & Innovation Framework Programme (CIP) and ICT policy support programme (http://ec.europa.eu/information_society/activities/ict_psp/index_en.htm)
- The eGEP measurement framework was illustrated by Luca M. Caldarelli: eGovernment Economic Project dates back to 2005 and provided a measurement framework as a tool for comparing an economic model and the corresponding expenditure study. The monitored parameters included efficiency, democracy and effectiveness indicators, and the lessons learned include difficulties in impact measurement, stressing the need for different stakeholders for different evaluations.
- Presentation by Peter Röthig (WiBe-TEAM) about measurement methods like the comprehensive WiBe framework – the approach adopted by the Federal Administration in Germany and one of the first worldwide.
- Presentation by Stephen Jenner, Director Ministry of Justice, UK government about eGov Portfolio metrics – the approach adopted to the Criminal Justice System IT Portfolio.
- Eric Velleman, of WAB-Cluster, illustrating the UWEM methodology as well as the available UWEM tools on the net.
- Jeremy Millard, of the Centre for Policy and Business Analysis, Danish Technological Institute, talked about measurement and illustrated the approach linking eGovernment measurement to the policy life-cycle, suggesting to focus on local rather than national targets.
- Relators from leading actors in the IT consulting market, like Capgemini (Barbara Loerincz) and Deloitte (Diane Whitehouse), addressed the positive effects of the shift of focus from a priori assumptions to the quality as perceived by the end users: in that sense, absolute quality is not the same as user satisfaction.
As a conclusion, we can say that it was a very good, running start for eGovMoNet, whose next events may build on this very productive kick off.
eGovMoNet: measurement of user satisfaction of e-Government solutions
The second meeting of eGovMoNet (the European network focusing on strategies to measure user satisfaction of e-Government solutions) took place in Copenhagen on October 24-25, 2008. WiBe-TEAM PR is involved in this European thematic network as partner since 2007.
Meeting report: Copenhagen Businesses School, DK – 24-25 Oct 2008
The second initiative was conceived to take one step forward the results of the 29-30 May 2008 Geneva kick-off meeting of eGovMonet, and in particular to:
- Present developments since the kick-off meeting.
- Introduce the first thoughts on how to describe the methods to compare them.
- Present methodologies in use according to the first measurement template.
- Discuss similarities and differences of the methodologies.
- Start to elaborate a set of key indicators for eGovernment impact and user satisfaction.
- Introduce a project handbook and hints on how to use the project work site.
- Offer a summary of project status and an update project progress plans and review status of project indicators.
As usual, there was plenty of presentations. We propose some of the highlights of this second step towards eGovMoNet maturity: it will be very evident that confidence continues to build as the network hones its tools and its ability to harmonize them.
- Stephen Jenner talked about The UK Central Government perspective about eGovernment and Customer Satisfaction measurement, illustrating the service transformation programme that – starting from the vision (let the service meet the needs of people rather then the needs of government), defining the strategic themes (including benchlearning, rationalizing, grouping services in a rational way, linking local and central government) and defining the measures that will let the government engage in customer satisfaction tracking mechanisms.
- Chiara Mancini from Emilia-Romagna Region (Italy) illustrated the co-design methodology, a strategy to design on line public services. This approach is based on two macro-phases: one, profiling users by expected benefits and satisfaction on e-government services (by preparing and executing a Computer-Assisted Telephone Interviewing survey on 3000 citizens in Emilia-Romagna on expected benefits and user satisfaction of e-government services).
- Eleni Vergi from the Observatory for the Greek Information Society talked about the Mistery User Methodology (MUM), a form of participative evaluation in which citizens, as users of the services, are brought in as active players of the evaluation process: their feedback assists in making e-services more citizen/ business centric.
- Uros Pivk presentation was about Measuring e-Government user Satisfaction in Slovenia. Among the many interesting points of this case is the gap between expressed interest from potential users and the actual use of e-government information and services. Factors influencing that gap have not yet been explained.
- Charlie Wallin (Kommunforbundet Vasternorrland) talked about E-governance in Sweden and presented the main e-government initiatives and the role of VERVA, Swedish Administrative Development Agency, promoting e-government services and producing Swedish National Guidelines for Public Sector Websites.
- Miriam Braskova and Anton Lavrin (Technical University of Kosice, Slovakia) let the audience know about “Highlights from Measurement Practice in Slovakia”. After an overview of the state of art of eGovernment initiatives, the conclusions stressed the lack of methodology design applicable to all levels of eGovernment services. Country specific problems delaying the adoption of the national programme for information society were also addressed.
- In its Interactive Session entitled “Stakeholder Perspectives” Jenny Rowley (Manchester Metropolitan University, UK) promoted and facilitated the discussion among eGovMoNet participants. After an introduction two plenary sessions took place: “Group activity: Exploring Stakeholders and their Perspectives” and “Towards an Understanding of Stakeholder Interests”.
- In her complex and insightful presentation, Christine Mahieu, Business Analyst FEDICT, related about the Belgian practice about user satisfaction and impact measurement. After providing some historical background about “eGov Measurement Framework Design” implemented in 2008, Mahieu described the methodology based on quantitative and qualitative survey, stressing the importance of defining the aims (status assessment, impact) and the intended users of measurement results (policy makers, researchers, public).
- Peter Röthig (WiBe-TEAM), after his kick-off talk, illustrated the connection of the WiBe Economic Efficiency Assessment concept to ISO/IEC 9126 – Software Engineering Product Quality standards. After delving in some depth into both the standards, showing how WiBe 4.x takes into account both the monetary and non-monetary sense of economic efficiency, Peter Roethig illustrated the harmonization of ISO and WiBe approaches along the life cycle of the product, gradually shifting from the hands of “virtual users” (the planning phase) to 100% real users (the application phase). Along the time axis, the need for user feedback grows. The need of policy maker decision tables and of three user group specific satisfaction indexes (customer, satisfaction, co-administrations) was covered in the last part of the presentation.
- After the kick off talk, Deloitte contributed with a talk by Patrick Wauters, about the preliminary results of a “Study on user satisfaction and impact in EU27”. The standardized methodology was illustrated in detail. The differences in socio-demographics in the 10 selected Member States apparently did not influence the overall picture. eGovernment services are overall used by 76% of the sample, with a peak for “looking for a job” and for “declaring income taxes” services.
eGovMoNet: eGovernment Impact Measurement Workshop report
eGovMoNet is a European thematic network dealing with measurement strategies for the impact and the perceived quality of eGovernment services. The workshop focused on eGovernment Impact Measurement. Participants in the network are leading organisations from eGovernment measurement including government, benchmarking practitioners, application developers, and users.
Meeting report: Veszprem, HU – 26-27 Mar 2009
eGovMoNet third workshop was held in Hungary in the scenic location of Veszprem. The Meeting goals included:
- Update on the developments occurred after the previous workshops
- elaborate further on how to describe methods and comparisons
- present further methodologies in use according to the update measurement template
- review impact indicator
- discuss the proposed dissemination plan
- introduce the network objectives.
The presentations, very shortly summarized herein under, may be downloaded from http://www.epractice.eu/en/workshops/egovernmentimpact
The first session was dedicated to gathering data on impact measurement practice using the template:
- Rony Medaglia (CBS): “User satisfaction and impact measurement” (Denmark)
- Rune Halvorsen (NOVA): “eGovernment, user satisfaction and impact measurements in Norway” (Norway)
- Peter Röthig (WiBe): “WiBe 4.0 – Impact measurements” (Germany)
- Stephen Jenner (PS): “eGovernment Impact Measurement – a perspective from the UK”
- Eric Velleman (ACC): “Websites of Netherland eGovernment” (Netherlands)
- Gino Verleye (UGent): “A conceptual model for the assessment of user satisfaction on online public services(Belgium)
Mikael Snaprud described ePractice community history and activities, asking partners for comments and suggestions about eGovMoNet deliverables and coming actions, in particular to project handbook (it will be published in the ePractice portal), dissemination plan, participation to events in Malmo, 18 to 19 November 2009
Further presentations followed – the space available compels us to limit the citation to the presentation title:
- Jeremy Millard (DTI): “Second measure paper summing up the user satisfaction observations and proposals for future work” (Denmark)
- Christine Mahieu (FEDICT), Gino Verleye(UGent): “Development of an operational eGov reporting environment: methodological and practical issues and learnings” (Belgium)
- Sebastiaan van der Peijl (Deloitte): “Measurement of eGovernment user satisfaction and impact” (Belgium)
- Francesco Belgioioso (RSO): “The Benchlearning first year results: an intense experience of measurement” (Italy)
- Morten Goodwin Olsen (Tingtun), Annika Nietzio (FTB): “Impact indicators: State of the art survey”
Among the deliverables of eGovMoNet project, the so called “measure papers” have a central role. Jeremy Millard presented the progress on the second measure paper focusing User Satisfaction Measurement (this paper will be completed by June 2009).
- The Third Measure Paper focuses on Impact Measurement and will be lead by Gino Verleye. Participants in the group are Jeremy Millard and Mikael Snaprud with further authors to be included.
- The Fourth Measure Paper will be focused on Innovative eGov Measurement Methods and practices, to be lead by Xavier Heymans. Participants in the group are Stephen Jenner, Mikael Snaprud, Morten Goodwin Olsen with further authors to be included.
All measure papers are planned to be compiled into the Measure Handbook to be prepared as one of the finalising deliverables from the project. All eGovMoNet meetings are open for interested participants and announced on ePractice as soon as the agenda is stabilised.
eGovMoNet: Innovative eGovernment Measurement Approaches Workshop report
Meeting report: Gent, BE – 30 Nov -01 Dec 2009
The 4th meeting of the eGovMoNet project, dealing with innovative eGovernment measurement approaches, was held in beautiful Ghent, Belgium. The meeting was successfully organised as a joint event with the Benchlearning Study on “measuring online public service impacts”.
The presentations covered topics from eGovernment impact measurements, stakeholders related to eGovernment measurements, invited guest presentations, a Knowledge Café session, and some ideas on how to continue the network beyond the project. All presentations from the eGovMoNet meeting are available online here …
The main discussion results where achieved in the Knowledge Caf session covering areas in which innovation can take place, innovative cases, and how Web 2.0 approaches can contribute to enhance innovations. This report covers the results of these discussions and issues raised for the presentations. Thanks to the inspirational participation of the over 35 delegates the workshop was a successful event and brought the network an important step forward. The meeting was held from 30. November to 1. December 2009 and was hosted by the University of Ghent.
eGovMoNet: Innovative eGovernment Measurement Approaches Workshop report
Meeting report: Bruxelles, BE – 08 Apr – 09 Apr 2010