Data Engineer

As a Data Engineer, you will be responsible for designing, building, and maintaining our data infrastructure, which includes streaming pipelines for applications and data lakes. You will collaborate with cross-functional teams, including business intelligence, analysts, product managers, and software engineers, to ensure the availability, reliability, and scalability of our data systems

Responsibilities:

  • Leading the development and implementation of streaming and batch data pipelines to extract, transform, and load data from various sources into our data lakes/data warehouse.
  • Implement self-service tools in the data platform (orchestration tools and data quality tools)
  • Implement a data validation process to improve data quality in data pipelines
  • Monitor the data platform cluster (Kubernetes)
  • Implement a data observability process to monitor data pipelines
  • Implement data lineage to tracking data movement
  • Implement data dictionary, relationship and its interpretation
  • Developing and maintaining data processing and analysis frameworks
  • Ensuring data security, privacy, and compliance with relevant regulations and policies
  • Collaborating with data analysts to understand data requirements and perform database design based on data understanding and requirements with consideration for performance
  • Manipulating complex data from different sources (MySQL, PostgreSQL, MongoDB, Excel, CSV, and Google Sheets)
  • Monitoring data quality and troubleshooting data-related issues
  • Identifying and implementing process improvements to enhance the efficiency and effectiveness of our data systems

Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field
  • 3-5 years of experience in data engineering or related field
  • Experience with streaming and batch data processing, ETL, and data warehousing
  • Experience in Cloud-based data platforms (Azure, AWS, GCP, or others)
  • Experience in Git, Docker, BigQuery, Airflow
  • Experience with databases (e.g., MySQL, PostgreSQL, MongoDB)
  • Strong programming skills (SQL, Python)
  • Strong understanding of coding standards and testing
  • Experience in managing projects and leading teams
  • Have a growth mindset and a willingness to learn new things or share knowledge with others
  • Ability to self-learn
  • Understand Kubernetes is a plus
  • Understanding the data lineage concept is a plus
  • Understand data observability concept is a plus
  • Understand data mesh concept is a plus

"To apply for this opportunity, please send your resume"
Email : hr@skilllane.com
More info : 061-419-0701 (Team People)


ร่วมงานกับ SkillLane

กำลังโหลดข้อมูล...