Job Description:
Data Engineer
LOCATION: REMOTE
WHO WE ARE:
HData delivers a technology platform that makes it easy for anyone in the U.S. energy industry to file, explore, analyze, and leverage regulatory data. With AI and automation, analysts can easily and accurately uncover the decision-making data they need in seconds instead of spending days sifting through thousands of pages of regulatory filings.
HData, a Techstars company based in Birmingham, AL, is open to working with qualified employees fully remotely, regardless of their location within the United States.
ABOUT THE POSITION:
As we continue to enhance our modern data stack, we are seeking a talented Data Engineer to join our team. The successful candidate will play a key role in designing and implementing robust data pipelines, optimizing data models, and leveraging technologies such as Snowflake, dbt, and Dagster to ensure the integrity, efficiency, and scalability of our data infrastructure.
YOU’LL THRIVE HERE IF YOU:
- Conduct yourself with honesty, integrity, and respect in all your interactions, aligning with HData’s Core Values.
- Can communicate effectively and operate in a fast-paced, dynamic environment.
- Can build partnerships that move our business forward.
- See feedback or failure as motivation to learn and grow.
- Believe data-driven decision-making is the norm.
ABOUT THE POSITION:
As a Data Engineer at HData, you will be at the forefront of our data engineering efforts, contributing significantly to the development and optimization of our data infrastructure. Your responsibilities will span the entire data lifecycle, from ingestion and transformation to storage and analysis. You'll work on implementing and maintaining data pipelines, optimizing query performance, ensuring data quality, and collaborating with cross-functional teams to deliver valuable insights. This role requires a blend of technical expertise and problem-solving skills while driving best practices in data engineering.
RESPONSIBILITIES:
- Design, build, and maintain scalable and reliable data pipelines using Snowflake, dbt, Dagster, and Elasticsearch.
- Implement and maintain dbt models to transform raw data into insightful, analytical-ready datasets.
- Collaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions.
- Monitor and troubleshoot data pipelines, proactively identifying and resolving issues related to data ingestion, transformation, and loading.
- Conduct data validation and testing to ensure the accuracy, consistency, and compliance of data.
- Stay up-to-date with emerging technologies and best practices in data engineering.
- Document data workflows, processes, and technical specifications to facilitate knowledge sharing and ensure data governance.
- Optimize data model performance and query efficiency in Snowflake.
- Implement data quality checks and data lineage tracking throughout the data pipeline.
- Assist in capacity planning and resource optimization for data infrastructure.
- Contribute to the development of internal data tools and frameworks to improve team productivity.
- Participate in code reviews to ensure high-quality, maintainable code.
- Implement and maintain robust testing and quality assurance processes for data pipelines and models.
- Continuously optimize and tune the performance of data pipelines and queries across various systems.
- Collaborate on the development and maintenance of our generative AI and RAG solutions.
- Ensure data privacy and security best practices are followed throughout the data lifecycle.
WHO YOU ARE:
- Bachelor's degree in Computer Science, Engineering, or a related field. Equivalent work experience will also be considered.
- 3+ years of experience in data engineering, ELT pipeline development, and data modeling.
- Strong proficiency in SQL and experience with cloud-based data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
- Hands-on experience with modern data transformation tools (e.g., dbt) and workflow orchestration (e.g., Dagster, Airflow).
- Strong programming skills, particularly in Python, with experience in developing and maintaining data pipelines.
- Experience with Elasticsearch or similar search and analytics engines.
- Knowledge of cloud platforms (AWS, Azure, or GCP) and their associated data services.
- Familiarity with data streaming technologies (e.g., Kafka, Kinesis).
- Experience with version control systems, preferably Git.
- Understanding of various data modeling techniques for both operational and analytical use cases.
- Experience with data governance, quality frameworks, and data privacy practices.
- Familiarity with CI/CD practices for data pipelines.
- Basic understanding of generative AI, Large Language Models (LLMs), and Retrieval-Augmented Generation (RAG) concepts.
- Ability to adapt quickly to changing priorities and work effectively in a fast-paced environment.
- Excellent problem-solving skills with a strong attention to detail.
- Strong communication skills and ability to work effectively in cross-functional teams.
- Demonstrated ability to learn and adapt to new technologies quickly.
NICE-TO-HAVES:
- Previous experience in the regulatory compliance or energy industry.
- Experience with real-time data processing and analytics at scale.
- Experience with vector databases (e.g., Pinecone) and implementing RAG solutions.
- Hands-on experience with LLMs and platforms like OpenAI.
- Familiarity in data visualization tools (e.g., Power BI, Tableau, Looker)
- Familiarity with data governance frameworks and practices.
- Knowledge of data security and compliance requirements (e.g., GDPR, CCPA).
- Familiarity with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
BENEFITS:
- Medical, Dental & Vision Benefits
- Performance Bonus
- 401k Retirement Plan
- 401k Matching
- Equity Benefit Package
- Remote, Hybrid, & In-Office Friendly
- Flexible PTO
- Relocation Assistance
- Life Insurance
- Assortments of Discounts, Perks, and Deals
HData is committed to promoting equality, inclusion, and diversity. We’re an equal-opportunity employer of the brightest minds we can find — regardless of race, gender, age, religion, sexual orientation, or identity.
Join HData and be a part of our mission to revolutionize regulatory compliance and empower the U.S. energy industry with streamlined data operations and business intelligence to help us drive and deliver impactful solutions in a dynamic and challenging industry.