Job Description
Job Summary: We are seeking a skilled and motivated Data Engineer with strong expertise in PySpark, SQL to join our Data Engineering team. You will be responsible for designing, building, and maintaining scalable data pipelines and processing systems that support our business intelligence, analytics, and machine learning initiatives.
Key Responsibilities:
Develop and maintain scalable, robust data pipelines using Scala and big data technologies.
Work with large datasets from multiple sources to ingest, transform, and make data available for analytics and reporting.
Collaborate with Data Scientists, Analysts, and other engineers to understand data requirements and deliver efficient solutions.
Optimize ETL jobs for performance and cost.
Ensure data quality, governance, and consistency across all environments.
Monitor production jobs, troubleshoot issues, and ensure system reliability.
Implement best practices for data engineering, including code reviews, testing, and documentation.
Required Qualifications:
~ Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
~6+ years of experience as a Data Engineer or in a similar role.
~ Strong skills in PySpark and SQL (experience with functional programming is a plus).
~ Hands-on experience with big data tools like Apache Spark , Kafka , Hadoop , or Hive .
~ Proficiency in building ETL pipelines and working with structured and unstructured data.
~ Experience with cloud platforms (e.g., AWS , Azure , GCP ) and their data services.
~ Familiarity with version control systems (e.g., Git ), CI/CD, and DevOps practices.
~ Solid understanding of data warehousing and data modeling concepts.
Preferred Qualifications:
Knowledge of SQL and database systems such as Postgres, Redshift, or Snowflake.
Exposure to containerization technologies like Docker and orchestration tools like Airflow or Kubernetes.
Job Tags
Full time,
Similar Jobs
Stagecoach Inn & Restaurant
Prep Cook Job Summary:The Prep Cook is responsible for preparing ingredients and performing basic cooking tasks to assist chefs and line cooks in creating dishes according to the restaurants standards. This position plays a vital role in ensuring the kitchen operates...
Domino's Franchise
...mention, it's work experience you're going to use for a long time to come. You've had our pizza delivered to you, now help... ...possible to make, bake or take pizzas during the hungry hours of the day and night, part or full time. You'll have plenty of time left over for school...
Kaiser Permanente
...Project Management; Risk Assessment; Service Focus; Requirements Elicitation & AnalysisCOMPANY: KAISERTITLE: Clinical Consultant V - Medical SubspecialtiesLOCATION: Pleasanton, CaliforniaREQNUMBER: 1342231External hires must pass a background check/drug screen....
Jean Ann Ryan Productions
...or an ENG-1 or ability to complete a medical exam. Must have Yellow Fever Vaccination. Must have Valid Seaman's Discharge / Record Book. Must be able to submit demo reels / links to previous performance work. GERMAN LANGUAGE SKILLS are suggested for this position...
Bristol Bay Native Corporation
...Vista Innovative Solutions, LLC is seeking a Molecular Biologist to support the 711th Human Performance Wing at Wright-Patterson Air Force Base, Ohio. This role supports biological and medical research efforts focused on aeromedical risk characterization through molecular...