Job Description

Summary
We are seeking a highly skilled and motivated Research Data Engineer to join the Social Media Archive at ICPSR. SOMAR is the only archive that facilitates direct researcher access to social media platform data. As a Research Data Engineer, you will play a pivotal role in connecting researchers with platform data through private APIs and trusted data environments. For instance, we proxy access to Meta's Content Library API through a combination of AWS services. We are also expanding our data sources beyond Meta. We work with other teams at the University of Michigan (U-M) and platforms to create trusted research environments that enable researchers to access and analyze sensitive data in secure, privacy-protecting ways. You can learn more about what we do from the SOMAR Director's recent presentation to the Accelerating Research Using Data Meshes and Data Fabrics (ARDM-24) Workshop.

Some great reasons to come work with SOMAR:

Work remotely as long as you're available for 2-3 weekly Zoom meetings during Eastern time business hours and responsive on Slack.
Meet quarterly API update schedules with realistic timelines for implementation that don't require overtime.
Be eligible for various benefits from U-M (specific benefits depend on the structure of your appointment).
Join a diverse, inclusive team that includes folks marginalized by their identities, abilities, and races.
No direct user interaction or on-call shifts required.


This position is a one-year, term-limited position with possible extension based on need and funding.

Responsibilities*
Proxy access to external APIs within a secure, cloud-based data analysis environment, preferably using Python for Lambda functions.
Orchestrate Lambda, RDS, S3, CloudWatch, and API Gateway integrations.
Build and evaluate automated testing pipelines to ensure code efficiency and precision and reduce manual testing.
Develop and maintain high-quality codebase, ensuring proper testing, documentation, and version control practices.
Explore, evaluate, and integrate state-of-the-art tools, libraries, and frameworks to enhance the team's data access capabilities.
Participate in code reviews and provide constructive feedback to ensure code quality, performance, and maintainability.
Coordinate with the project team and external partners to establish and complete technical deliverables on schedule.
Required Qualifications*
Proven experience (3+ years) as a Data Engineer, Software Developer, or similar role, with a focus on data-intensive applications.
Strong Python programming skills with a solid understanding of software engineering principles and best practices.
Expertise with AWS console and prior experience with services such as AWS Lambda, API Gateway, CloudWatch, and CloudFormation.
Capable of maintaining a CI/CD pipeline.
Strong problem-solving and analytical thinking skills, with the ability to work independently.
Experience working with agile data warehouse design through rapid prototyping, iteration and validation.
Desired Qualifications*
Ability to navigate between strategic work and detailed execution.
Bachelor's degree or higher degree.
Certifications from AWS, Azure, and/or Google Cloud.
Experience with using SQL (e.g., Postgres) and ability to optimize data storage and retrieval.
Experience using, building, and configuring REST APIs.
Experience with version control systems (e.g., Git) and collaborative development practices (e.g., code reviews, continuous integration).
Prior experience in an academic or research setting.