Big Data Support Engineer

Urgent
Posted 6 months ago
Pune, Maharashtra
Application deadline closed.

Job Description

Big Data Support Engineer

Arminus

Pune, Maharashtra

Primarily Skills and Experiences Extensive experience in Hadoop Ecosystem Concepts Tools and Modules including Apache ZooKeeper Apache Solr Apache Ambari Spark Hadoop Application cluster YARN KAFKA Extensive experience in Hadoop Ecosystem architecture and HDFS Must be able to Manage and maintain Hadoop Ecosystem for uninterrupted job Check backup and monitor the entire system routinely Take end to end responsibility of the Hadoop Life cycle Handle the installation configuration and supporting of Hadoop Ecosystem Write Map Reduce coding for Hadoop clusters help to build new Hadoop clusters Manage and monitor Hadoop log files Ensure that the connectivity and network are always up and running Plan for capacity upgrading or downsizing as and when the need arises Manage HDFS and ensure that it is working optimally at all times Secure the Hadoop ecosystem in a foolproof manner Responsible for the documentation and architecture of Hadoop applications Maintain data security and privacy… Regulate the administration rights depending on the job profile of users Add new users over time and discard redundant users smoothly Have full knowledge of HBase for efficient Hadoop administration Be proficient in Linux scripting Advanced working knowledge and experience working with relational databases query authoring as well as working familiarity with a variety of databases previous experience working with PostgreSQL and MongoDB Experience building and optimizing big data data pipelines architectures and data sets Create and maintain optimal data pipeline architecture Assemble large complex data sets that meet functional nonfunctional business requirements Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition operational efficiency and other key business performance metrics Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement Strong analytic skills related to working with unstructured datasets Build processes supporting data transformation data structures metadata dependency and workload management Develop data mining architecture data modeling standards and more A successful history of manipulating processing and extracting value from large disconnected datasets Working knowledge of message queuing stream processing and highly scalable big data data stores To design web applications for querying data and swift data tracking at higher speeds Experience supporting and working with crossfunctional teams in a dynamic environment