Data processing and analytics solution
Discover how we built a real-time web-based election data processing solution that speeds up election ballot chart building by 200 times.
March 2021 – present
About the client
Live Vote Results (name changed for NDA purposes) is a site that provides real-time election results by electoral area. The site’s workers upload data from official social surveys and elections to the admin panel. Then the service converts the data into structured charts.
To get data for ballot charts, Live Vote Results polling station commissioners manually uploaded data from social surveys and elections to the website. The client’s objectives were to:
- reduce the operating time for manual data processing
- ensure website scalability and stable performance under high loads
- secure election results from possible fraud
- optimize the website’s technology stack and user experience
- deploy the website by the April elections
Reducing manual work and optimizing operational processes
Generating 64,000 ballots for a single area would normally require 600 hours because of the amount of manual work required to check CSV files with data and build ballot charts. To optimize this process, we automated business rules via scripts that cover all manual work and produce fast, error-free results.
Optimizing the technical solution
We performed a detailed analysis of functional and nonfunctional requirements in search of a well-balanced technical solution. We implemented a GraphQL API for interacting with multiple data objects, as it predefined the software architecture and supplied a broad range of templates. The Aurora MySQL database has data field limitations but won us 25% more time for software development.
Ensuring website scalability and high-load capabilities
Live Vote Results needs to show many views of filtered ballot charts, so we had to ensure the site could easily scale. We chose AWS Lambda and AWS AppSync for the GraphQL API, as they offer on-demand scalability. With these technologies, our client can add servers to power up the site for election time and disable unnecessary servers at other times. We also conducted load testing of Live Vote Results. The site can easily handle over 100,000 requests per second, while 450 users can simultaneously request filtered ballot charts and get them in less than two seconds.
Providing anti-fraud mechanisms
To avoid mistakes or intentional manipulation of ballot chart results, we added a logging feature to the admin panel and implemented different roles for it. Polling station commissioners only have access to the area they’re responsible for, while the admin panel automatically logs their work. If any incorrect information is found in a ballot chart, an admin can check the logs and roles to see who was the last to edit the chart and ask them to correct it.
Improving the website UX
Our client required us to add interactive maps to help users filter election results and ensure a certain level of website customization. For this, we implemented:
- interactive maps. Our client supplied us with detailed district maps. With the help of React, we processed maps in the SVG file format and created clickable maps that let users view election results for a selected area.
- chart customization. To prioritize election results based on a user’s interests, we added chart order customization. We also allowed polling station commissioners to preview charts and chart lists before publishing. Whenever required, polling station commissioners can configure electoral district names and sizes as well as create new districts.
We continue working on the project. For the first release, we managed to deliver a fully functional interactive website by the April elections, and it worked flawlessly. With process automation, we reduced the operational effort required for data processing and visualization.
BEFORE: 600 worker-hours
Manually read and apply voting rules
Manually fill out ballot charts
Upload files to the election machine
AFTER: 3 hours’ work of the admin panel
Run a script to apply voting rules
Run a script to generate ballot charts
Upload files to the election machine
CONFIGURE REAL-TIME DATA PROCESSING AND MANAGEMENT
Yalantis will develop a solution capable of analyzing and visualizing a large amount of data quickly and seamlessly.