Evan Byrne

Overview

According to the Southern Poverty Law Center, "The SPLC is a catalyst for racial justice in the South and beyond, working in partnership with communities to dismantle white supremacy, strengthen intersectional movements, and advance the human rights of all people."

The Southern Poverty Law Center (SPLC) is a nonprofit organization with a storied history of doing exactly what they claim. They were a major force of good during the civil rights movement, notably bankrupting the Ku Klux Klan by litigating on behalf of victims, and the organization continues to fight extremism in America. A core aspect of their operations is tracking the presence of hate groups. To do this, they have a massive multi-service software application codenamed Dossier, which is used for targeted web scraping and analysis.

Implementation

Dossier was already legacy software by the time I was tapped by colleague Natasha Shields to oversee its migration. SPLC needed to move Dossier from their previous vendor and had selected Amazon Web Services (AWS) as their new home. My job was to design a plan of attack, provision infrastructure for development and production environments, provide guidance on next steps, and eventually take over management of the engineering team.

Dossier is a series of Java and JavaScript services connected with a Kafka cluster. At the time, the application was backed by a 14TB MongoDB database, and had 41 different source code repositories. Unfortunately, the previous vendor had written zero documentation for this fully custom collection of software, so the first challenge was simply getting all of the various services built and running in a development environment. After that, Terraform was used to provision infrastructure on AWS. Finally, Jenkins was set up to build services and then trigger Ansible to complete deployments.

Results

The successful migration of Dossier to AWS allowed SPLC to continue to monitor the online activities of violent extremist groups. Following the migration, I took on the position of engineering management and oversaw a team on behalf of SPLC. Together, we provided functional improvements, security updates, and recommendations to increase the lifespan of Dossier even further. Without the work my team performed over many years, Dossier would have ceased to operate, and SPLC would have lost one of their most valuable tools for combating hate.