Rotterdam
Enablers
1.1 - Skills
Does the local authority have internally any position such as service designers OR user researchers OR user experience experts? Evidence: 25
The employees assigned in such positions include the UX lab staff and experts (3 UX experts and 2 UX/service designers), WCAG experts/testers. We have multiple people assigned as designers within DEvOps teams etc. CV's or organization schedules available when desired |
Yes |
Has the local authority provided training on service design or user research to its employees in the last three years? Evidence: Hundreds (via innovation network training, specialists training, plus we have the 20 person designer pool in Rotterdam) |
Yes |
Has the local authority provided training on ICT to its employees in the last three years? Evidence: Thousands (on privacy, security. MS apps, etc) (feb 2023: still applicable: mandatory training apply for numerous aspects of ICT such as security and privacy aspects, and numerous applications) |
Yes |
Has the local authority provided training on ICT to citizens in the last three years? Evidence: strictly speaking this is done under municipal funding and direction by a separate partner: www.rotterdam.hetinformatiepunt.nl. the number mentioned was 3292 in 2021, numbers 2022 not yet available. |
Yes |
1.2 - Strategies
Does the local authority have a digital strategy less than three years old? Evidence: https://www.rotterdam.nl/wonen-leven/rotterdam-digitaal/
update: still applicable 2023 |
Yes |
Does the local authority have a position of Chief Digital Officer or equivalent? Evidence: https://nl.linkedin.com/in/basboorsma?challengeId=AQEaQKoZQATU6AAAAYBlppLu4NgpiuYidlgC5jzgG8H_6zUx0CPcAw0tPF7kZzM-buxUF6bavKqV-zEozvEvoiTnBH1eQSh8VQ&submissionId=8a3c6dd2-106f-e916-e6f1-4b5367a2e343 (Update 2023: still applicable) |
Yes |
Does the local authority have formal service standard (like for instance the UK Government Digital Service standards)? Evidence: Update 2023: Standards have changed from service standards to "citizens satisfaction". Internally though, may quality related items are measured, monitored and reported, such as: waiting times, response time website/social media, call-back time, etc. The reason for this change in approach lies in the fact that there is only limited correlation for service standards with citizen satisfaction, our main goal. |
Yes |
Does the local authority have design guidelines valid across departments, including for instance standards or protocols for simple language? Evidence: Specifically for Simple Language, a whole team is working on this (NB, this is an intranet page): https://rio.rotterdam.nl/Project/Duidelijketaal. furthermore we have a WCAG compliant cross-departmental guideline for (digital-) design (Rotterdam DesignSystem RODS as presented in Rotterdam to UCC community, see ppt provided. Read also: https://www.rotterdam.nl/toegankelijkheid
|
Yes |
Does the local authority have in place formal methods to monitor and enforce such service standards and design guidelines? Evidence: There are different dashboards applicable to this item, all used to monitor and govern our standards, for instance:
Net Promoter Score
System Usability Scale
powerBI dashboard
site improve
etc...
Dashboards are only accessible from within our network. Screenshots available on request
|
Yes |
Are service design or user experience mentioned in digital strategy or some other strategy level document? Evidence: https://www.watdoetdegemeente.rotterdam.nl/apps/voorjaarsnota/programmas/bestuur-en-dienstverlenin/burgerzaken/Programma-Dienstverlening-2019-2022.pdf
We sent the NEW programme as a pdf-file to UCC. The programme is called De Rotterdamse Dienstverlening, which roughly translates into “The public services of the City of Rotterdam”. In this policy paper, we don't literally mention the words service design and/ or user experience. This is intentional. Because these words clearly aren't common language for our citizens, nor widely used within our organization (yet). In the pdf-file, we highlighted and translated the relevant parts in which policy refers to service design and/ or user experience. The city Counsil has yet to approve this strategy, it is sceduled for debate soon.
This comment also applies to the current programme although we chose to highlight the passages in the NEW programme rather then the old one because it better reflects our commitment the upcoming 4 years |
Yes |
1.3 - Ecosystem
Does the local authority provide APIs to other administrations and to private companies? Evidence: Yes we do. Data and API’s are provided through the municipal open data platform, which can be accessed here: https://rotterdam.dataplatform.nl/#/data
An example of a provided API: https://rotterdam.dataplatform.nl/#/data/6a58bf9f-62b4-4d15-aed1-e3368c46418e?tabName=download_tab&tabId=3 |
Yes |
Does the local authority use standardised services modules, provided at national or European level (e.g. CEF building blocks, national payment service or eID)? Evidence: https://dloket.rotterdam.nl/loket/burger/digid-akkoord?dest=%2Fburger%2Fstartform%3Fid%3DDV_VHZ&auth=DIG1
national standards for payment is used (Ideal for Netherlands https://www.ideal.nl/en/) and DigiD for identification as a national standard: https://www.digid.nl/en and many others) |
Yes |
Has the local authority carried out within the last two years innovative forms of procurement, such as pre-commercial procurement, hackathons, other forms of collaboration with SMEs and startups? Evidence: Innovative procurement is done on average in every 10 EU-procurement procedures. Recent examples are what we procured for our digital twin. evidence available on request, these are internal documents |
Yes |
User-Centricity Performance
2.1 - Co-creation
Does the local authority habitually (as standard practice on every new digital service) use service co-design / user research sessions in developing its services before their launch? Evidence: This can be divided in 1. commissioned service design (privately owned studios), 2. internally commissioned research e.g. by the UX lab, and 3. initiatives by individual project managers in cooperation with service design experts.
1. in 2022 approximately 280.000 euro has been spent by Rotterdam on service design assignments. The number might be much higher due to the fact that not all procurement's contain the words "design" or "servicedesign".
2. approximately 25 assignments per year, for instance www.rotterdam.nl website testing with citizens, chat010 chatwidget co-creation and testing process
3. An example https://www.linkedin.com/posts/bjorndirkse_videocalls-design-contextmapping-activity-7027197720100491264-eSZX?utm_source=share&utm_medium=member_desktop
Before the launch of the new website, more than 10.000 citizens participated in tests of the beta version, resulting in valuable feedback. It seems that Rotterdam is the first in the Netherlands to have chosen this co-creation strategy for the main website. Press coverage: https://openrotterdam.nl/nieuwe-website-gemeente-rotterdam-voor-en-door-rotterdammers/
|
Yes |
Does the local authority habitually carry out users' research sessions on a regular basis (at least once a year) AFTER the launch of a service? Evidence: Of course our public services use the reviews and responses of the public to make iterations and changes to existing services. One of the ways this is done is by the UX lab. The UX lab tested about 15 applications in 2022, among them some internal, but also multiple external applications such as the website. At least 7 citizens are involved per assignment. documentation available. The UX lab's turnaround in 2022 was around 156k, in 2023 we expect to double this. The UX lab provides testing, co-creation and other forms of design related activities both before as after launch of a |
Yes |
Does the local authority habitually use specific service co-design / user research sessions with disadvantaged communities (e.g. minorities, elderly, disabled people, etc.)? Evidence: Disadvantaged citizens are taken into account when developing public services in which it is required to do so. We work with for instance https://www.lezenenschrijven.nl/ and https://www.meerotterdamrijnmond.nl/. We have articles and pictures of such processes available on request. In 2022 700 municipal employees have been trained in using simple language to provide better service and communication for citizens with limited language skills. evidence available on request. |
Yes |
Does the local authority habitually use web analytics and other service data to improve digital services (e.g. completion rates and most frequent errors)? Evidence: We use dashboards (Cognos) to show the number of calls, the topics, the service level norms, etc We use siteimprove to monitor errors, dead links, behavioural mapping etc. super cool stuff. screenshots available on demand. |
Yes |
Does the local authority habitually plan for releasing regular (at least once a year) new releases for existing digital services (not including technical updates automatically provided by software provider)? Evidence: Multiple feedback loops, such as customer signal framework, UX testing etc leed to a constant process of iteration and improvement of services. Is it habitual: absolutely. planned and regular: we prefer to work agile and devops, which means we do whats needed, when needed, and (citizens) needs driven, which doesn't allow for a strict schedule or plan. We can deliver evidence of customer signal framework analysis on request, and more info on this customer feedback and agile improvement loops. For example: the Digital Counter (launched in 2020) will be updated in march to version 7.0. |
Yes |
2.2 - Supply of online services
Does the local authority provide the majority of services fully online (out of total services provided that could potentially be digitalised)? Evidence: https://dloket.rotterdam.nl/loket/burger/digid-akkoord?dest=%2Fburger%2Fstartform%3Fid%3DDV_VHZ&auth=DIG1
After logging in you can access all relevant sources.
Another platform is: https://www.rotterdam.nl/
due to national policy (Plasterk 2017) it is mandatory for local authorities to provide public service digitally/online where possible. A "maturity" scan on Rotterdam products (406 in total) shows that only 25 of those are physical service only. 75 products are (also) digitally available on level 1, which means "application by email or PDF form" and the others (306) are fully digital if desired by the citizen. This leads to a percentage of almost 94% is available digitally, 75.4% is available fully digital. Evidence available on request from internal dashboard. |
Yes |
Does the local authority have a dedicated app that is downloaded by more than 20% of the population? Evidence: We have various dedicated apps (Rotterdampas, Digitale Balie, MeldR, etc). Most recent apps are not built as native apps but more modern PWA or webapps. This does not require downloads, but behaves like a native (downloadable) app. PWA or hybrid app have advantages such as more possibilities for swift iteration. The use of these apps vary because current state of affairs is that we have dedicated apps, not 1 fits all app.
There is not one single app used by 20% of the population. The reason for this is in the initial answer: The city of Rotterdam has a different, more agile and contemporary strategy on app-provision. “Native apps” will rarely be added to app stores, because of the benefits of responsive website-apps as explained. Because we have different apps for different targetgroups or goals, they are never used or downloaded by 20%+ of the population. So basically you could say the answer should be changed to “NO” although in our opinion this wouldn’t reflect the service provision through apps by Rotterdam. I can not support this with data before the deadline, but it is likely that 20%+ of the population has ever used ANY app by the municipality, depending on what definition is used for the word “app”. Therefor we recommend this dashboard item to be reviewed maybe. (but we can live with scoring a “no” as well ;-) |
NA |
Does the local authority offer at least one proactive service, where users are automatically signed up for a service based on the government-held data? |
NA |
Has the local authority already fulfilled the requirements of the Single Digital Gateway (deadline end of 2022)? Evidence: We are compliant to phase 1, phase 2 is work in progress. https://www.waarstaatjegemeente.nl/Dashboard/dashboard/landelijke-voorzieningen
This dutch national website website works like an app, the url stays unchanged for any selection. If “Rotterdam” is selected on the site, it provides a dashboard of where every municipality in the Netherlands stands on different legislation and compliance, among them are the SDG guidelines (bottom right). The dashboard is curated by the VNG, the Dutch organization for municipalities, and provides audited data. Screenshot sent to UCC for evidence |
Yes |
Does the local authority provide habitually to citizens the possibility to check online the progress status of the services they request? Evidence: Citizens have access to their own portal, https://www.rotterdam.nl/loket. Behind the security wall (inlog with DigiD, dutch digital ID) they find the progress of their service request. screenshots available on demand |
Yes |
2.3 - Useability
Does the local authority habitually carry out useability assessment of its online services, using standard tools such as System Useability Scale (SUS)? Evidence: SUS is standard methodology in almost all UX research processes. The Digital Counter measures SUS for most of the videocalls by automatically sending survey links to the citizens who used the service. Evidence available on request through our dashboards serviced by Expoints. |
Yes |
Do all the websites or websites sections of the local authority have consistent design and look&feel? Evidence: For all digital content and applications the use of Rotterdam Digital Design System is mandatory for all creators, both internal and external. https://www.rotterdam.nl/huisstijl provides all guidelines for look and feel (online and offline as well as AV for instance) with more specific on UX: https://www.figma.com/file/ZWSC4gCrOXRUR9UX3aoZ8x/Gemeente-Rotterdam-Design-System?node-id=413%3A13001 |
Yes |
Does the local authority provide the possibility to citizens to have live audio/video interaction (such as videoconference or single telephone number)? Evidence: Digital Counter (Digitale Balie) is the standard for public service through video interaction: https://www.rotterdam.nl/videobellen
Video calls, access to relevant digital processes including the right authentication methods etc. The Digital Counter is the most advanced videoplatform for local governements available, and invented and developed by the municipality. The SUS score rates about 78, NPS 35 avarage.
1 central phone number/point of entry: 14010 https://www.rotterdam.nl/bellen |
Yes |
Are the local authority web services in line with accessibility guidelines (WCAG)? Evidence: Most web services are, evidence: https://www.rotterdam.nl/toegankelijkheidsverklaringen. some (mostly newer or innovative) applications are under continuous testing and auditing to require the necessary certification.
|
Yes |
2.4 - Security and privacy
Are the users able to use national eID, as a means of authentication for online services requiring authentication? Evidence: for instance see https://dloket.rotterdam.nl/loket/burger/digid-akkoord?dest=%2Fburger%2Fstartform%3Fid%3DDV_VHZ&auth=DIG1 where citizens can register their change of address themselves. the website asks for EID (DigiD in NL), as is for all online services. |
Yes |
Has the local authority put in place measures to ensure citizens' control over the data held about them (such as seeing who has access to the data and for what reason, correcting data, etc.)? Evidence: See https://www.rotterdam.nl/uw-gegevens
|
Yes |
Has the local authority in place documents on measures, practices or procedures on ICT security? Evidence: We have a CISO, who is in charge of making sure our security is both as locked as possible, as well as usuable as possible at the same time (balancing act). We take a lot and serious care of our security (not just technical, social as well). In the Demand Supply procedures Security is by default one of the checks before development or procurement starts. Development is security and privacy by design. The development department has standards for security checks, penetration tetst and procurement of external expertise through onboarding bootcamps etc. lots of documentation available on demand, internal documentation. |
Yes |
2.5 - Citizens redress and feedback mechanisms
Does the local authority provide online mechanisms for both citizens and businesses to complain and seek change to a decision? Evidence: Online you can leave feedback: https://www.rotterdam.nl/klacht-indienen, or one can call us (14010) anytime and you can use an app to complaint about public spaces https://www.rotterdam.nl/meldr. Text mining technology scans all qualitative feedback on the complaint-channels, which is reported through the appropriate channels. The website mentions many ways to complain or give feedback https://www.rotterdam.nl/zoeken?zoekopdracht=klachten and theres a feedback button for website feedback and on all applications. A complaint will be coordinated by a central complaints-coordinator (1FTE), and can be referred to a departmental coordinator, available in every department. |
Yes |
Has the local authority put in place structured means for users to provide feedback? Evidence: Among other means, the most evident example is the feedback button on websites and applications. for instance see www.rotterdam.nl (allow cookies) |
Yes |
2.6 - Interoperability
Does the local authority apply the once-only principle in its services, so that citizens are not required to submit documents already held by the administration? |
NA |
Does the local authority encourage the use of standards in IT development and procurement? Evidence: Most important standard is the common ground strategy which Rotterdam supports: https://commonground.nl/ Common Ground is a national ambition to get to a common architecture to make sharing, integration and connectability easier. Rotterdam also supports the national policy to encourage open source software development. This plays an important role in for instance the development of the Digital Twin: https://www.rotterdam.nl/digitale-stad, but also participation in chatbot GEM https://commonground.nl/groups/view/978adf62-97f2-4955-87cc-7fee0cd238e0/team-virtuele-assistent-gem |
Yes |
Has the local authority policies in place to encourage the use of open source solutions? Evidence: see above |
Yes |
Outcome
3.1 - Adoption
Considering only services that are available online, are the majority of transactions carried out online? Evidence: Considering only services available online, then transactions of those carried out online as well will be close to 100%. For evidence also see reported number of digital services. more evidence (from internal dashboards) available on request |
Yes |
Does the local authority publish data on usage of online services (such as a dashboard with metrics on transactions) on a regular basis? Evidence: We use dashboards (Cognos) to show the number of online services etc. This is an internal dashboard. We also have the open data platform mentioned earlier: https://rotterdam.dataplatform.nl/#/data |
Yes |
3.2 - Reduction of burden
Does the authority measure the average time saved by citizens when using an online service compared to the offline one? |
NA |
Does the authority measure the amount of annual financial savings for the public administration? |
NA |
3.3 - Satisfaction
Does the local authority measure the citizens’ level of satisfaction with regards to the services’ provision? Evidence: Nps is used on many channels, but not all digital services. The burden for citizens needs to be minimal, so we selectively measure. There is a lot of data on NPS measurements on for instance phone, Digital Counter, Website and other services available on requests. This is available from within our network. Evidence available on request |
Yes |
Is the share of satisfied users above 80%? Evidence: In 2022 we measured the satisfaction through NPS on different channels. Reporting over 2022 is still in progress and has not been published, but the number is 84,6%, measured on channels phone and counter/physical visits. Chat and video calls are excluded. From 2023 we will measure the percentage of not-satisfied citizens, as a governement target. This means the definitions might be deviating in the future. Evidence (data) is available on request, from internal dashboards and datasets. |
Yes |
Background Information
655473
Total population
13669
Total number of employees in the city/region administration
3900000000
Total annual budget (last year or average of the last three years)
800
Number of ICT specialists employed directly (full-time equivalent)
300
Number of ICT specialists employed indirectly (subcontracting/outsourcing)