digitalmktg
Automating Process Flow Increasing Accountability
June 30, 2016
telecom
P&L Application TMF Guideline Implementation
July 5, 2016
UKgovt

Bringing Scattered Data to One Platform Ease of Making Decision

Bringing Scattered Data to One Platform Ease of Making Decision

Summary

The purpose of this project was to bring the data scattered across 180 databases to one single platform for providing a holistic view. Our client, a leading IT company, provides services to UK government for its operation in more than 180 countries using different applications.For each country a separate database was being managed and maintained at different data centers across the globe. Reporting mechanism was in place for each country to provide Management with parameters which can help them to make effective decisions. The Reporting mechanisms, however, failed to achieve the purpose of providing complete and comparative view of each country’s operations with respect to other countries; as the number of reports was specific to a country and the reporting was spread across 23 such reports for each country. Having clear insights as to where we stand in terms of organization objective and taking corrective measures to move towards achieving it was only possible when data of all these different countries could be consolidated and analysed on different parameters. This was currently not possible in the existing situations. Client realized the need and engaged our team for the project.

We analyzed all 180 databases and studied underlying schema structure in record time of 15days. We designed a single schema structure which can accommodate the need of all 180 databases. We implemented scheduled tasks running at regular intervals to import data from different countries databases to 3 central repository databases using Python as the scripting language. We created web interface to facilitate the client for generating parameter based report. Client was happy as the implemented solution provided him consolidated views across countries, improved efficiency and reduction in expenditure cost.

Client Profile

A leading IT company which is part of a 2-billion-dollar group and specialize in handling Government Projects

Business Situation

Customer was running 180 databases, 25 application connecting these databases, 30 persons engaged for managing reports for 40 countries, with 30% delay and 5% errors. No solution available with the Management team to run report based on parameters. Instead Management team had to look out for reports from the emails.

Key Challenges for Sagacity

  • Customer was running 180 databases, 25 application connecting these databases, 30 persons managing reports for 40 countries, with 30% delay and 25% errors.
  • Central repository of data from all 180 databases.
  • Reports for various country and various departments had to based on parameters, and this was not available.
  • Ensure 99% uptime of reporting system.
  • 98% percent accuracy of reports was expected.
  • First set of reports had to be ready within 3 months after the project had been awarded
  • Historic report can be run at any point of time, current system was running report just in time and was not reproducible with same data.
  • Need to be very careful in handling the data due to data sensitivity and applicability of data protection act.
  • Engaging teams across different countries for defining and implementing process for data transfer.

Sagacity Solution

We analyzed all 180 databases and studied underlying schema structure in record time of 15days. 3 central database repositories were created based on application version which was runningat various data centers. Customized ETL application was developed to collect data from various database.Jasper report tool has been used for report development.AWS cloud has been used toensure horizontal scalability and ease of sourcing infrastructure. File based data transfer was developed to ensure fast processing of data at file system level. Data is growing with rate 3 GB per day and processing time was critical as window for processing data was only 4 hours.

Impact

  • Central repository gave a single view to management about all centers and performance of each center
  • Accuracy of reports has increased to 99%
  • Uptime of Management reporting achieved to 99% as per customer requirement
  • Historic report can be executed at any given date for last one year
  • Any failure triggers alarm well before in time for team to take appropriate action before report fails
  • Web based solution, now instead of checking email user can view reports from anywhere and anytime
  • Man power reduced from 30 persons to only 3 persons saving huge operational cost for government
Share
Facebook
Google+
https://sagacitysoftware.co.in/casestudies/bringing-scattered-data-to-one-platform-ease-of-making-decision/
Twitter
Instagram