Architecting A Data-Centric Self Service BI Model
A national health data and analytics provider was facing an uphill battle in providing a scalable solution to their customers, due to significant growth in customer data demands and the sheer volume of the underlying data. Sagence identified areas in the reporting layer that dramatically improved customer experience and manageability of the application. Sagence also implemented a solution that reduced their backend data processing time and data redundancy, thereby reducing operational costs (estimated to be millions of dollars over the next few years).
A new self-service reporting and analytics platform was developed and deployed to a small number of subscribers. As usage expanded the performance degraded, which jeopardized subscriber contracts and consumer confidence. After several attempts, the organization realized the need for external expertise to coordinate the troubleshooting and remediation across the many different parties involved.
Concurrently, the client’s operational cost hike became more urgent due to recent service changes with the database vendor. This health data and analytics provider was looking for ways to streamline daily data processing to reduce inefficient use of storage space. This challenge also exposed the organization’s need to reign in internal user’s ability to replicate data for siloed needs and highlighted the importance of a holistic data management strategy at the enterprise level.
The first phase was a rapid assessment that yielded immediate actions to ameliorate the performance issues. Our team quickly realized that the majority of the issues stemmed from leveraging software components in ways they were not designed for (something we see quite often). We proposed and assisted with the implementation of data and application changes that returned the performance back to satisfactory levels.
The second phase was the planning, design and implementation of architectural changes that promoted a data-centric service model, and ensured the flexibility and scalability of the reporting and analytics platform. We provided architecture expertise in several areas, including data, application, integration and security. We implemented a data processing strategy that allows incremental data loading — this reduced load time and need for storage space, a solution that resolved the issue brought on by ever increasing data volume and storage costs.
Lastly, we performed a data virtualization proof-of-concept to prove out a viable option to manage diverse internal data needs without unruly data replication that is both, expensive and error prone due to data versioning issues.
In the short-term, the client was able to satisfactorily address the performance issues, thus retaining its subscriber base and avoiding a negative brand impact.
In the mid-long term, our recommendations guided our client towards a data-centric service model where the data is the product delivered through multiple channels. This service model enabled our client to increase its subscriber base by allowing data to be consumed in its “raw” format or using a range of reporting and analytics applications. Our recommendation also dramatically reduced operational costs, improved backend data processing performance and better served internal data needs in a more disciplined manner; this removed the distractions caused by outdated architectural designs and in turn enabled our client to be more scalable and competitive in the healthcare data provider business.