- Design, architect, and build stream processing systems which deal with unbounded data
- Design, build, and deploy software using technologies such as Kafka, Mqtt, Spark, Serverless, Kubernetes.
- Work with the data engineering team to develop & maintain data pipelines for batch & stream processing.
- Work closely with Development Architects on designing/implementing code optimizations.
- Identify, design, and implement internal process improvements and tools to automate data processing and ensure data integrity while meeting data security standards
- Work with data scientist and business analytics team to assist in data ingestion and data-related technical issues
- Perform in-depth analysis of performance issues to identify hotspot and opportunities for optimization
- Provide technical consulting and mentoring to team members on optimization techniques, data processing design, implementation and conformance to architecture standards.
- Minimum bachelor's degree in Computer Science, Applied Mathematics, Engineering, or any other technology related field. An equivalent of the same in working experience is also accepted for the position.
- Minimum 8 years of software development and system integration experience including 4 years of experience in data warehousing / distributed system (e.g. Hadoop)
- Good Java / J2EE, Python, Scripting experience
- Proficient in Big data tools and ecosystem (e.g. Hadoop, HIVE, HBase, Kafka, Spark etc)
- Expert in building and optimizing big data pipelines, architectures, and data sets
- Experience in data ingestion, cleaning and processing tools.
- Excellent experience in data processing using Scala/Python/Java
- Good in user requirements gathering, maintenance, and support
- Highly organized, self-motivated, pro-active, and able to plan
- Experience managing users and vendors is a plus
- Past hands on experience with Advanced Analytics is desirable
- Good Architectural concepts around Cloud Computing and base software Development
Interested candidate, please submit your updated resume in MS WORD format to: email@example.com
We regret that only shortlisted candidates will be notified.
Name: Yong Whei Jie
Registration Number: R1110096
EA Licence Number: 02C3423
Octavius, Whei Jie Yong EA License No.: 02C3423 Personnel Registration No.: R1110096