Required Skills:
Client is looking for a BI developer who possesses the following skills: data warehouse design and development, Spark ETL/ELT development, Airflow DAG development, and data mapping and analytics.
Desired Skills:
- Languages:
– Python including PySpark
– T-SQL / SQL - Apache Spark, Apache Airflow, and related technologies • Microsoft SQL Server
- Cloud Technologies such as Azure and AWS
Responsibilities and Duties:
Develop ETL and ELT processes using various data sources including from/to SQL and Oracle databases and Apache Spark Data Lake and Delta Lake
Develop and maintain Python code (PySpark) for Apache Spark data lake and Delta Lake management and queries