Big Data Engineer
Experience: 2-5 Years
Job Location: Mumbai
Job Description:
- Responsible for Hadoop development
- Implementation including loading from disparate data sets, preprocessing using Hive and Pig.
- Scope and deliver various Big Data solutions
- Ability to design solutions independently based on high-level architecture.
- Manage the technical communication between the survey vendor and internal systems
- Maintain the production systems (Kafka, Hadoop, Cassandra, Elasticsearch)
- Collaborate with other development and research teams
- Building a cloud based platform that allows easy development of new applications
- Experience with Java oriented technologies (JBoss, Spring, SpringMVC, Hibernate, REST/SOAP)
- Experience with non-relational & relational databases (SQL, MySQL, NoSQL, Hadoop, MongoDB, etc.)
- Experience with Spark, or the Hadoop ecosystem and similar frameworks
- Familiarity with various tools such as AWS, Mesos or Docker and an instinct for automation