Big Data Engineer

Title: Big Data Engineer

Location: Denver, CO

Duration: 12+ Months

Job Description:

• Deploy Enterprise data-oriented solutions leveraging Data Warehouse, Big Data, and Machine Learning frameworks
• Optimizing data engineering and machine learning pipelines
• Support data and cloud transformation initiatives
• Contribute to our cloud strategy based on prior experience
• Understand the latest technologies in a rapidly innovative marketplace
• Independently work with all stakeholders across the organization to deliver point and strategic solutions

Skills - Experience and Requirements

• Should have prior experience in working as a Data warehouse/Big-data.
• Experience in advanced Apache Spark processing framework, spark programming using Scala or Python with knowledge in shell scripting.
• Coding experience in Java and/or Scala is a must.
• Experience in using AWS APIs ( e.g., JavaAPI, Boto3, etc.) to integrate different services
• Should have experience in both functional programming and Spark SQL programming dealing with processing terabytes of data
• Specifically, this experience must be in writing Big-data data engineering jobs for large scale data integration in AWS. Prior experience in writing Machine Learning data pipeline using a Spark programming language is an added advantage.
• Advanced SQL experience including SQL performance tuning is a must.
• Experience in logical & physical table design in Big data environment to suit processing frameworks
• Knowledge of using, setting up, and tuning Spark on EMR using resource management frameworks such as Yarn or standalone spark.
• Experience in writing spark streaming jobs (producers/consumers) using Apache Kafka or AWS Kinesis is required
• Should know a variety of data platforms such as Redshift, S3, DynamoDB, MySQL/PostgreSQL
• Experience in AWS services such as EMR, Glue, Athena, IAM, Lambda, Cloud watch and Data pipeline
• Experience in AWS cloud transformation projects is required.

Plexova Solutions provides a complete range of consulting services to help businesses accelerate their technology deployments. We offer a comprehensive view of technologies like cloud, mobile, and security.