Posted on
Strong in Python/Pyspark scripting, minimum 3+ yrs., (Focus on Python/Pyspark and relax on snowflake)
Must have hands on experience implementing AWS Big data lake using EMR and Spark.
Working experience with Spark, Hive, Message Queue or Pub/Sub, Streaming technologies (3+ years)
Have 6+ years of experience developing data pipelines using mix of languages (Python, Scala, SQL etc.) and open source frameworks to implement data ingest, processing, and analytics technologies.
Experience leveraging open source big data processing frameworks, such as Apache Spark, Hadoop and streaming technologies such as Kafka.
Hands on experience with newer technologies relevant to the data space such as Spark, Airflow, Apache Druid, Snowflake (or any other OLAP databases).
Experience developing and deploying data pipelines and real-time data streams within a cloud native infrastructure preferably AWS Experience in using CI/CD pipeline (Gitlab) Experience in Code Quality implementation (Used Pep8/Pylint) tools or any other code quality tool.
Sky Solutions, LLC is a niche services company that provides next generation solutions to enterprises who want to improve their business outcomes. These next-gen solutions are created using the digital platforms the marketplace is investing in already. By taking this approach, it provides focus as to where and how Sky Solutions can future proof your business through its Domain Expertise, Technology, and Staff Augmentation capabilities while also allowing for additional return from your software investments.