Staff Data Engineer
HealthVerity
Full-time
On-site
Philadelphia, Pennsylvania, United States
$70,000 - $200,000 USD yearly
How you will help
As a data engineer on the data platform team, you will be supporting and enhancing the platform that supports HealthVerity’s Petabyte-scale core data asset. You will work closely with other engineers, data scientists, and business leaders to ensure that our data platform is available, secure, and reliable. You will use your strong engineering and product mindset to understand business needs and develop scalable engineering solutions that support HealthVerity’s product roadmap and vision while continuously looking for opportunities to simplify, automate tasks, and build reusable components.
What you will do
- Engineer efficient, adaptable and scalable data pipelines to process structured and unstructured data
- Develop and maintain data pipelines to efficiently process and analyze large amounts of streaming data
- Collaborate with other data engineers to maintain a cohesive and standardized data infrastructure
- Work closely with the software engineering team to integrate data pipelines into the overall platform architecture
- Collaborate with cross-functional teams including software engineers, data scientists, product managers, and analysts to understand data needs and deliver valuable platform enhancements that support the overall HealthVerity vision and roadmap.
- Identify and implement solutions to optimize data storage, retrieval, and processing
- Continuously evaluate and improve data engineering processes and systems to increase efficiency and scalability
- Stay up-to-date with emerging technologies and industry trends in data engineering
- Ensure data security and compliance with privacy regulations
- Troubleshoot and resolve data-related issues in a timely manner
- Leverage large-scale distributed computing and serverless architecture including Spark, AWS Lambda, etc. to develop pipelines for transforming data
- Partner with the product teams to understand product goals and provide data that enables us to respond to customer and regulatory data requests
- Monitor data quality and proactively identify and resolve data issues
How Success is defined
Required skills and experience
- You are proficient in at least one primary language (e.g., Java, Scala, Python) and Advanced SQL (any variant)
- You have a product mindset to understand business needs and develop scalable engineering solutions.
- You are always looking for opportunities to simplify, automate tasks, and build reusable components across multiple use cases and teams.
- You have strong communication skills to collaborate with cross-functional partners and drive projects. You are curious and eager to work across a variety of engineering specialties (i.e., Data Science, Data Engineering, and Machine Learning to name a few).
- You have an eye for detail and like to spark joy amongst your partners with well-documented high-quality data products that are modeled and easy to understand.
- You are able to successfully lead large, complex systems design and implementation challenges independently
- You have a strong knowledge of Databricks features and functionalities, such as Unity Catalog, Audit Logs, Databricks SQL and Delta Live Tables
- Experience with CI/CD pipelines and DataOps
- Experience using Infrastructure as Code (IaC) tools, such as Terraform, YAML, and Helm Charts
- You have 8+ years of industry experience and proficiency in building distributed data pipelines for both batch and real-time (experience with Databricks, Spark, Hive, Iceberg, Kafka, Snowflake is helpful, but not strictly required).
- You have experience with Databricks pipeline automation, AWS EMR, AWS S3 service, AWS Services Snowflake, Spark, Docker
Base salary for the role is commensurate with experience and can range between $70,000 - 200,000 + annual bonus opportunity.
Hiring Locations
Approved States: CT, DE, FL, GA, IL, IN, MA, MD, MI, NC, NJ, NY, OH, PA, RI, TN and VA.
We prioritize hiring team members from the Philadelphia area whenever possible, as our main office is located in Center City, Philadelphia, where we operate on a hybrid model with in-office work required two days a week. However, for certain roles, we’re also open to hiring from our hub locations, where we have groups of remote workers. Please note that, due to tax and labor regulations, we can only hire from specific states, but remote work is supported in the following key hubs and approved states:
Hub Locations:
- Philadelphia, Pennsylvania
- Boston, Massachusetts
- New York City, New York
- Baltimore, Maryland
- Washington D.C.
- Charlotte, North Carolina
- Raleigh-Durham, North Carolina
- Atlanta, Georgia
- Chicago, Illinois
Approved States: CT, DE, FL, GA, IL, IN, MA, MD, MI, NC, NJ, NY, OH, PA, RI, TN and VA.
About HealthVerity
HealthVerity synchronizes transformational technologies with the nation’s largest healthcare and consumer data ecosystem to power previously unattainable outcomes and fundamentally advance the science. We offer a comprehensive, yet flexible approach, based on the foundational elements of Identity, Privacy, Governance and Exchange (IPGE), that synchronizes unparalleled Identity management with built-in Privacy compliance and Governance, providing the ability to discover and Exchange a near limitless combination of data at a record pace. Together with our partners in life sciences, government and insurance, we are Synchronizing the Science. To learn more about HealthVerity, visit healthverity.com.
Why you'll love working here
We are making a difference – Our technology is at the forefront of some of the biggest healthcare challenges in the world.
We are one team – Our people define our culture and always will. We take time out to celebrate each other at the end of every week through company-wide shout outs, and acknowledge the value that each of us adds towards our greater mission. Come share all you have to offer.
We are learners – Every team member is continually learning, no matter if we've been in a role for one year or much longer. We are committed to learning and implementing what is best for our clients, partners, and each other.
Benefits & Perks
- Compensation: competitive base salary & annual bonus opportunity (for non-commissioned roles)
- Benefits: comprehensive benefits with coverage on Day 1, medical, dental, vision, 401k, stock options
- Flexible location: our HQ is in Philadelphia. We offer both hybrid roles and those with quarterly travel.
- Generous PTO: Take time off as needed, targeted at 4 weeks per year, including vacation, personal and sick time, plus paid parental leave.
- Comprehensive and individualized onboarding: mentorship program, departmental talks, and a library of resources are available beginning day 1 for each new team member to minimize the stress of starting a new job
- Professional development: biweekly 1:1s, hands-on leadership that is goal-and growth-oriented for each team member, and an annual budget to support professional development pursuits
HealthVerity is an equal opportunity employer devoted to inclusion in the workplace. We believe incorporating different ideas, perspectives and backgrounds make us stronger and encourages an environment where ageism, racism, sexism, ableism, homophobia, transphobia or any other form of discrimination are not tolerated. All qualified job applicants will be given consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability. At HealthVerity, we’re working towards an innovative and connected future for healthcare data and believe the future is better together. We can only do that if everyone has a seat at the table. Read our Equity Inclusion and Diversity Statement.
If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please direct your inquiries to careers@healthverity.com
Remote opportunities are not available in all areas and require team members to work from a fixed location due to tax and labor law implications - specific questions about remote positions can be discussed during the interview process with your recruiter.