Published: Jul 09, 2024

empowering data-driven decision making to support Singaporeans’ skills mastery

SkillsFuture Singapore (SSG) supports millions of Singaporeans in their pursuit of skills mastery. Recognising the critical role of swift data access and insightful analytics in providing Singaporeans with the opportunities to develop their fullest potential throughout life, SSG embarked on a transformative partnership with NCS. Over a period of 12 months, NCS migrated the existing data environment and analytics platform from a constrained on-premises legacy system to a scalable cloud-based analytics solution. This allowed SSG to unlock significant cost savings and new potentials in data agility to support the evolving needs of Singaporeans in their upskilling and reskilling journeys.

Seeking a cost-effective, scalable data analytics environment

SSG envisioned a platform where data is highly tailored and democratised. This comprises of a platform that is highly accessible across departments, user-friendly, and adept at generating rapid insights to address a spectrum of business challenges.

However, SSG’s system greatly restricted the creation of a democratised data analytics platform. Maintenance was cumbersome and inefficient with a system that struggled to scale alongside a growing user base. The diverse array of data ingestion scripts, each written in a different programming language, only compounded the complexity.

NCS and SSG collaborated to transform SSG’s use of data within the organisation by adopting a more robust and efficient infrastructure on Government on Commercial Cloud (GCC2.0), integrating solutions from AWS, Talend and Tableau. The transition to a cloud-based analytics solution allowed SSG to quickly scale and adapt, ensuring that it remains at the forefront of supporting individuals’ upskilling and reskilling journeys.

Leveraging cloud to democratise data analytics for insight-driven decision making

Confronted with the critical need for a data environment that is scalable, secure, and highly available, NCS migrated SSG’s on-premises data analytics environment and analytics models onto AWS GCC2.0. The move to cloud opened a myriad of possibilities for SSG. NCS also worked with SSG to strengthen data governance, improve data democracy, and enabled self-service data visualisation capabilities.

The project was navigated with meticulous precision - working closely with the technical teams from AWS and GovTech to design the architecture, and developing a robust, multi-phased approach. It successfully translated to the completion of the migration and system enhancements within a period of 12-months. 

To ensure users’ smooth transition and usage of the solution, the NCS team also developed comprehensive transition instructions, tailored training programs, and change management materials. This holistic approach not only facilitated immediate adoption, but also empowered SSG’s team to harness the full potential of the enhanced system from day one and beyond.

Key highlights of the solution delivered to SSG:

• Migrated SSG’s data analytics environment and analytics models onto GCC2.0 to effortlessly scale with demand and support core data functions across multiple departments. The on-premises data analytics environment was refactored onto AWS GCC2.0 – leveraging on solutions such as AWS RedShift, Amazon Athena, etc.
• Designed and implemented an MLOps Pipeline on AWS Sagemaker, to streamline, automate and monitor the migrated analytics models throughout their lifecycle.
• Converted the 3,000+ data ingestion scripts into a configuration-based fast data ingestion framework on Talend Big Data Management which eliminates the need to develop and maintain multiple data transformation scripts, allowing for quick and easy data on-boarding.
• Strengthened data governance by implementing metadata management and data cataloguing to allow users to quickly locate data by searching in business terms; and data lineage to visualise the flow of data.
• Strengthened existing data governance within SSG to enhance data quality and communication within the organisation. NCS collaborated with key SSG stakeholders to set-up automated workflows that assisted in streamlining data ingestion processes and governance.
• Ability to quickly analyse data with complex SQL queries through serverless Amazon Athena and Redshift Query Editor V2 without worrying about the underlying infrastructure.
• Empowered SSG users with self-service analytics and visualisations through user friendly platforms with drag-and-drop interactivity.

Revolutionising the use of data to foster a data-driven environment

Migrating and transforming the then on-premises system onto GCC2.0 not only allowed SSG to democratise data to support their daily operations; it revolutionises the way data is harnessed and empowers SSG to foster an environment where data-driven decisions are the norm, not the exception. 

The collaboration between SSG and NCS catalysed the creation of a data analytics environment that was not only exponentially faster but also more secure and robust, while at the fraction of the cost. Thus, enabling stakeholders to derive meaningful insights with unprecedented clarity and speed. 

• 93% of SSG users surveyed are satisfied with the enhanced data analytics environment in supporting their daily operations. With 91% finding the comprehensive support resources from NCS to be significantly useful for self-support.
• On-demand access to data analytics within SSG by implementing intuitive, and easily customisable solutions. Thus, enabling both technical and non-technical users to leverage on a full suite of analytic tools for strategic decision making. 
• Reduced time to ingest new data sources into the data environment for analysis from weeks to days, providing heightened agility and accelerated access to new data sources.
• Improved system availability to 99.5% by moving from on-premises legacy system onto a cloud-based data environment and analytics model.
• Increased compliance in infrastructure patching and reduced maintenance turnaround time by moving from on-prem to cloud-native services.