Indian Parliamentary Data

The challenge

Indian political archives span dynamic web pages, long audio recordings, PDFs, and inconsistent institutional names. The work had two distinct phases: first determining which speech and translation tools fit the research workflow, then engineering the collection and processing pipeline end to end.

Studio Lab: evaluation phase

At CU Boulder’s Studio Lab, my primary work was comparative research. I designed word-error-rate evaluations across Deepgram, Google Speech-to-Text, Amazon Transcribe, and Azure Speech, then set up parallel translation comparisons across Google Cloud Translation, Amazon Translate, and Azure Translator. I also researched configuration and language-adaptation options for political names and domain-specific vocabulary.

The goal was not to promote one vendor. It was to understand how standard tools behaved on the project’s material and identify which components fit the workflow.

Princeton: engineering phase

Under the later Princeton appointment, I built the scraping logic and orchestrated the workflow end to end. That included dynamic Parliament Digital Library pagination, resumable SQLite state, speech and translation processing for more than 40 GB of 2014 and 2019 campaign audio, PDF text extraction with PyMuPDF, and structured outputs for further analysis.

I parsed thousands of parliamentary statements and used fuzzy matching to map inconsistent ministry names into a standardized research representation. Checkpoints and intermediate artifacts made long-running jobs restartable and kept transformations open to review.

Engineering decisions

• Evaluate interchangeable services before embedding one vendor into the research workflow.
• Persist collection state so interrupted sessions can resume near the failure rather than restarting.
• Keep raw source material, intermediate text, and normalized outputs distinct so transformations can be inspected.
• Treat source text and fuzzy matching as assistive normalization inputs that still need review, not unquestioned ground truth.
• Separate the campaign-audio corpus from the parliamentary-document corpus so scale and date claims are not merged.

Research context

The evaluation work began under a paid CU Boulder Studio Lab appointment from January 21 through April 13, 2024. After that appointment ended, I continued collaborating on the research. Princeton later paid me directly under a separate appointment beginning January 15, 2025 for the end-to-end engineering phase; that appointment concluded in May 2026. The roles are listed separately because the employers, appointments, and contribution phases were distinct even though the research project continued across them.