The paper describing this work can be found at https://isprs-archives.copernicus.org/articles/XLVIII-4-W12-2024/43/2024/
This work proposes a framework for cloud platforms to carry out a self-assessment from a user experience perspective. The study outlines the methodological approach used to develop the framework, including the classification of existing platforms through a literature review of EU-funded projects. Additionally, a survey on user experience was conducted to further inform the design of the framework. We consider as target users the developers of policy-relevant use cases. The final phase of the methodology identifies user requirements across ten dimensions within the development lifecycle and suggests a range of heuristic principles to facilitate self-assessment. The paper emphasises the need to address the identified challenges and limitations to improve the usability and user satisfaction of existing cloud platforms. It also highlights the importance of principles such as FAIR (findability, accessibility, interoperability, and reusability) and TRUST (transparency, responsibility, user focus, sustainability and technology), open source components, and Open Geospatial Consortium (OGC) standards in shaping the future of data platforms and infrastructures.
The heuristic principles that we identified are grouped into focus areas and described in the following table
| Dimensions | Indicators | Description |
|---|---|---|
| Interoperability | compliance with standards | adherence to established data exchange standards, such as from OGC or ISO |
| data interoperability | cloud services providing access and integrating datasets from heterogeneous data sources | |
| model interoperability | integration and use of different models or algorithms for sharing and collaboration | |
| service interoperability | integration of external services or applications | |
| semantic interoperability | use of standard vocabularies, ontologies, or semantic technologies for consistent and meaningful data interpretation | |
| Access | of data | availability, variety, data types, granularity, resolution, coverage, timeliness, frequency at which new data is added, authentication, authorization, licensing and usage rights (clear information on data usage restrictions, copyright, and intellectual property rights), quality (accuracy, reliability, and completeness), metadata |
| of models | availability of pre-built models or algorithms | |
| of services | availability of up-to-date services (such as GPUs, HPC, real-time updates, continuous data streams, large storage capacity, No-Code or Low-Code AI solutions, containerisation, data cubes, MLOps, DevOps). Seamless access through federation of services. | |
| user diversity | compliance with accessibility design principles and standards, e.g., Web Content Accessibility Guidelines (WCAG) | |
| Discovery | integration with discovery services | services to search for spatial datasets and related applications |
| data and service metadata completeness | complete and accurate metadata to enable discovery | |
| analytics | data and analytics to assess how users discover and access the services | |
| search engine optimisation | optimisation of service-related pages and information for discovery services and search engines | |
| service catalogues/directories | service catalogues/directories listing and categorising available services | |
| Documentation | completeness | documentation covering all aspects of the infrastructure |
| frequency of updates | access to regular updates of the documentation | |
| coverage of use cases | documentation covering various use cases to guide users in different scenarios | |
| searchability | easiness in navigating relevant documentation | |
| demos and media | availability of notebooks, videos, tutorials, webinars | |
| try before buy | free trials/freemium-like models | |
| Customisation | data visualisation | customisation of data visualisation |
| data processing | customisation of data processing and analysis tools | |
| workflow customisation | users can customize their workflows, ideally accessing resources from a federation of resources, in a seamless way | |
| roles and permissions | customisation of user roles and permissions | |
| integration | integration of the platform with external systems or services chosen by users | |
| Customer support | response time | time needed to answer to user inquiries or issues should be customized based on the level of operationality |
| multilingual support | support in different languages | |
| support team availability | availability of the support team to assist users during specified hours or through different channels (e.g., email, chat, phone) | |
| support team expertise | level of expertise and knowledge of support team should be paired to user’s issue avoiding unnecessary communication and wasting of time, e.g. initial triage and ticket assignation (possibly assisted by AI) | |
| support process improvement | continuous improvement of support processes based on user feedback and internal evaluations | |
| Community | community engagement | engagement and participation of the community members |
| knowledge sharing | knowledge, insights, and experiences sharing, adherence with FAIR, TRUST, and GEO principles | |
| collaboration opportunities | forums, discussion boards, or project collaboration features | |
| community growth | growth rate of the community | |
| diversity and inclusivity | presence of different user profiles, backgrounds, and expertise, creating an inclusive and diverse community, availability of a code of conduct | |
| user support and mentoring | support and mentoring provided by experienced community members | |
| Cost Transparency | tranparent prices | clarity and transparency of the pricing structure, which should be easily accessible, understandable, and communicated |
| cost breakdown | cost breakdown of the different cost components, such as data access fees, processing fees, or additional charges | |
| cost predictability | providing users with a clear understanding of the costs they can incur for using the services. Thresholding, pre-paid, or other mechanisms to prevent unexpected charges | |
| no hidden costs | absence of hidden or unexpected costs. All costs disclosed upfront | |
| cost comparison | services to compare costs with alternative service providers or different pricing plans of the platform. informed decisions for users based on cost comparisons | |
| cost optimisation recommendations | recommendations or insights to optimize their costs, tools or suggestions to help users reduce their expenses while using the services | |
| Sustainability | revenue growth | generation of increasing revenues, capacity to attract and retain customers. Proxy for platform's longevity |
| churn rate | rate at which customers leave the platform over a specific period, sustainability of the business model in terms of customer loyalty | |
| commitment | Commitment of service provider over an agreed period of time | |
| innovation and adaptability | ability to innovate and adapt to changing market dynamics and customer needs | |
| carbon footprint | GHG emissions due to platform operations | |
| renewable energy use | share of used energy from renewable sources | |
| power usage effectiveness | measure of a data centre efficiency, as total energy divided by IT energy; values close to 1 indicate high efficiency | |
| Performance | responsiveness | platform uptime, data retrieval and processing speed |
| scalability | ability to handle increasing data and model sharing demands | |
| security and privacy | robust data protection, privacy, security measures that protect users' data, with clear and transparent privacy policies (e.g., data encryption, access controls) | |
| data use | number of downloads, user interactions, data usage |
If you are a user of cloud platforms, please participate to our survey and help us validate the indicators that we identified.
Link to the presentation.