Java Pyspark Technical Lead

New Today

Must Have Technical/Functional Skills MUST HAVE: Scala, Java, Pyspark, Spark, BigQuery, Dataproc NICE TO HAVE: Event Engine, Cloud composer, Shell scripts, Hadoop(MapR), Hive, Scheduler Minimum 4+ years of experience in Implementing data migration program from Hadoop (On-prem) with Scala, Java, Python, Spark to GCP BigQuery, Dataproc Minimum experience of 4+ years in Integrating plugins for GitHub Action to CICD platform to ensure software quality and security. Experience of 4+ years on Google Cloud platform (GCP) tools mainly BigQuery, Dataproc, Cloud composer, Cloud storage, etc., Minimum 4+ years of experience in Cloud Deployments. Configuring scripts in Cloud console / Jenkins for Automated executions Prior work experience in BFSI Warehouse functional domains will be an added advantage. Hands on experience on Pipeline creation with Scala scripts from scratch and troubleshooting cloud configuration issues. Should have worked on Git and Continuous Integration environment like GitHub action, Jenkins Should have experience SQLs, NoSQL Databases Should have experience in working in teams following Agile or XP methodology. Should have BFSI domain experience to understand the business requirement. Minimum 2+ years of Senior programming level experience involving some architecture and high-level design. Understanding of distributed systems and related concepts required Roles & Responsibilities Participating in requirement gathering sessions Meeting with Client business analyst to understand requirements. Collaborating with Solution Architects and Design teams for implementing data pipeline migrations. Deploying Data pipeline in GCP, custom Scala, Java, Python scripts, BigQuery on Google Cloud Platform for Bigdata scripts Configuring and executing projects Test scripts using TESTNG framework Using DevSecOps tools available on JENKINs and Google Cloud platform Participation in SCRUM meeting and sprint grooming sessions. Identifies areas for process improvements. Perform proof of concepts to evaluate technical fitment of frames works technologies for suitability. Participate in the ongoing migration roadmap technical skills to review, verify, and validate the software code developed in the project and troubleshooting techniques and fix the code bugs
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Location:
Phoenix, AZ, United States
Category:
Computer And Mathematical Occupations

We found some similar jobs based on your search