Jobs for People with MS: National MS Society

Mobile National MS Society Logo

Job Information

Nielsen Sr. Data Engineer - (Big Data, Spark, Scala, Python, AWS, RDBMS, SQL) in Bangalore, India

At Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when you succeed, we do too. Embark on a new initiative, explore a fresh approach, and take license to think big, so we can all continuously improve. We enable your best to power our future.

Responsibilities

  • Build scalable, reliable, cost-effective solutions for both the Cloud and on-premises with an emphasis on quality, best-practice coding standards, and cost-effectiveness

  • Build and test Cloud-based applications for new and existing backend systems to help facilitate development teams to migrate to the cloud.

  • Build platform reusable code and components that could be used by multiple project teams.

  • Understand the enterprise architecture within the context of existing platforms, services and strategic direction.

  • Implement end-to-end solutions with sound technical architecture, in Big Data analytics framework along with customized solutions that are scalable, with primary focus on performance, quality, maintainability, cost and testability.

  • Drive innovative solutions within the platform to establish common components, while allowing customization of solutions for different products.

  • Develop design specifications, continuous build and deployment strategy to drive Agile methodology

  • Setup and manage expectations with consultants engaged in the projects.

  • Provide cloud integration development support to various project teams.

  • Build rapid technical prototypes for early customer validation of new technologies

  • Collaborate effectively with Data Science to understand, translate, and integrate methodologies into engineering build pipelines

  • Collaborate with product owners to translate complex business requirements into technical solutions, providing leadership in the design and architecture processes.

  • Provide expert apprenticeship to project teams on technology strategy, cultivating advanced skill sets in application engineering and implementing modern software engineering practices

  • Mentor junior team members, providing guidance and support in their professional development

  • Stay informed about the latest technology and methodology by participating in industry forums, having an active peer network, and engaging actively with customers

  • Cultivate a team environment focused on continuous learning, where innovative technologies are developed and refined through collaborative effort

Key Skills

  • Domain Expertise

  • Bachelor’s degree in computer science, engineering plus 4-8 years of experience in information technology solutions development.

  • Must have strong cloud Implementation expertise in cloud architecture.

  • Must have strong analytical and technical skills in troubleshooting and problem resolution.

  • Must have the ability to provide solutions utilizing best practices for resilience, scalability, cloud optimization and security.

  • 3+ years of experience: big data using Apache Spark in developing distributed processing. applications; building applications with immutable infrastructure in the AWS Cloud with automation technologies like Terraform or Ansible or Cloud Formation.

  • Technical Skills

  • Experience in software development using programming languages & tools/services: Java or Scala, Big Data, Hadoop, Spark, Spark SQL, Presto \ Hive, Cloud (preferably AWS), Docker, RDBMS (such as Postgres and/or Oracle), Linux, Shell scripting, GitLab, Airflow, Cassandra & Elasticsearch.

  • Experience in big data processing tools/languages using Apache Spark Scala.

  • Experience with orchestration tools: Apache Airflow or similar tools.

  • Strong knowledge on Unix/Linux OS, commands, shell scripting, python, JSON, YAML.

  • Agile scrum experience in application development is required.

  • Strong knowledge in AWS S3, PostgreSQL or MySQL.

  • Strong knowledge in AWS Compute: EC2, EMR, AWS Lambda.

  • Strong knowledge in Gitlab /Bitbucket .

  • AWS Certification is a plus

  • "Big data" systems and analysis

  • Experience with data warehouses or data lakes

  • Mindset and attributes

  • Very strong verbal and written communication skills

  • Advanced analytical and technical skills in troubleshooting and problem resolution.

  • Ability to coach, mentor and provide guidance to junior colleagues

DirectEmployers