What are the evaluation criteria for Country Gateways success ?

-- Oleg Petrov (opetrov@worldbank.org), September 08, 2000

Answers

Evaluation will be based on the analysis of thematic on- and off-line surveys, analysis of user comments for trends and common issues, as well as the publicity of the project. An important indicator of CG success will be the number and variety of Country Gateways forums as well as their activity and discussion quality. The long-term impact of Country Gateways will be evaluated by the scope of created partnerships and effectiveness of on-line communities.

-- Oleg Petrov (opetrov@worldbank.org), September 08, 2000.

Constant evaluation Country Gateway's activities is essential for its successful implementation. Evaluation will be done via analysis of automatically generated metrics (using special software), thematic on- and off-line user surveys, analysis of user comments for trends and common issues, as well as the publicity of the project.

Below is a list of the criteria that will help to measure the success of the Gateway. During the startup period, the Gateway team will further develop and establish qualitative targets in line with these indicators.

Publicity / Influence on development issues:

Number and range of participating members, partners and donors

Number of unique users

Number of people aware about the project and its purposes

Citation index in the Internet Community (e.g., Alexa.com Internet Traffic Ranking)

Citation index in mass media

Evaluation of a Country Development Gateway by every community (NGO, government, businesses) via special surveys on such issues as "Influence of Gateway on trade volume through e- procurement," "Whether opportunities for funding of development activities have increased or decreased?", "Whether transaction costs in aid and international business became lower?" etc.

Quality review of user comments/rating

Content evaluation:

User surveys on such issues as access to development information, easiness to navigate (the quality of guides and design), exchange/transfer of development knowledge and lessons, etc.

Number and range of users visiting particular platforms

Content population on the particular topics

Creation of new topics / subject matter specialists

Turnover of subject matter specialists

Re-use of original content, knowledge

Target audience evaluation

Gateway User Surveys/ Automatically generated statistics on such issues

Number of unique users

Age groups Gender Education User Sessions by Geographical Location Technology Evaluation / Survey (more appropriate for Global Development Gateway if present technology model will be realized) Usage trends by time Number of unique users Number of user sessions Number of page view

Paths traveled including entry and exit

Stickiness of site (clock, repeat visits)

User evaluation of such issues as design, functionality of a portal, effectiveness of search engine, etc.

Turnaround time for problem resolution

On the later stage, it is important to include qualitative economic and development evaluation criteria for Gateway activities, such us:

Number of projects submitted by NGOs, private companies and organizations through the Gateway and supported by donors; amount of money raised from donors/foundations through country gateways;

Number of best practices in e-government, e-SMEs, e-commerce, other e- development areas identified within country gateway projects and implemented/replicated elsewhere;

Amount of investment, increased sales, new jobs created resulting from Country

Gateway's implementation;

Number of partnerships built among stakeholders as a result of participation in Country Gateway projects;

Investment attracted to country/region, amount of procurement deals, decreased costs of government operations/services, time saved by citizens thanks to interacting with government agencies on-line - resulting from e-government initiatives within country gateways;

-- Oleg Petrov (opetrov@worldbank.org), September 25, 2000.