Beschreibung
· Data Engineering
AWS
ETL data warehousing
DevOps
EC2, S3, RDS, EMR, Redshift
Deep understanding of AWS services relevant to data platforms, including EC2, S3, RDS, EMR, Redshift, and Glue.
· Experience with data engineering and data warehousing concepts.
· Proficiency in designing scalable and fault-tolerant architectures.
· Design and architect the overall AWS infrastructure solution for the Customer Data Platform (CDP), ensuring it aligns with business requirements, scalability, security, and cost-effectiveness.
· Define the high-level architecture, including network topology (VPC), compute resources (EC2 instances), storage solutions (S3, EBS), and security controls (IAM, Security Groups).
· Collaborate with DevOps engineers to create and maintain infrastructure as code (IaC) templates using tools like Terraform or CloudFormation
· Oversee the implementation of AWS services, such as EC2, S3, RDS, Lambda, and others, to support the CDP’s data ingestion, processing, and storage needs.
· Develop and maintain documentation for the infrastructure architecture, including diagrams, deployment procedures, and best practices.
· Ensure compliance with AWS security best practices and industry standards, including data privacy and protection regulations.
· Monitor and optimize the infrastructure for performance, availability, and cost