Cloud/Data Engineer

Company: Triumph Services
Location: Rockville
Posted on: November 20, 2020

Job Description:

This is a DIRECT HIRE opportunity with an excellent benefits package including bonuses and equity. This position will start off remotely due to COVID-19, but will transition to onsite at some point in the future. At that point, there is still some flexibility for occasional remote work. MUST HAVES - AWS, SPARK, and Data Lake experience. Please do NOT apply to this position if you do not have all of the above skills and experience. NICE TO HAVES - Scala, R, Python, Indexes We are seeking a highly motivated Cloud Data Engineer to join a small, highly collaborative team and help create the next generation of our platform. Our team is fast-paced, and where we solve a wide variety of interesting technology challenges using modern Cloud technologies. Yoursquoll work on real-world problems in a dynamic environment with talented team members to deliver reliable and scalable applications. At the heart of that technology is a cloud-based data lake to support research and development. If you're excited about working with cutting-edge technologies like containerization, microservices, and infrastructure automation, this is the position for you Responsibilities Optimize data lake performance by identifying and resolving production and application development problems. Act as Data Steward, ensuring Data Governance best practices with developers. Help investigate and resolve data anomalies including data quality issues and ambiguous data definitions. Recommend data integrity checks and controls to ensure data quality. Define datainformation architecture standards, structure, attributes and nomenclature of data elements, and applies accepted data content standards to technology projects. Develop data retention practices and system lifecycle for our index applications based on our data retention policies. Hands-on ability to set up reporting tools and build reports and ad hoc functionality, giving usersrsquo access to their data. Experience with master data management, data governance, data security, data quality and related tools desired. Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional non-functional business requirements Identify, design, and implement internal process improvements automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS lsquobig datarsquo technologies. Work with QA Test analyst to ensure test coverage (Including Integration Regression testing) Develop new program logic andor assembles standard logic modules to create new applications Qualifications Experience Good understanding of Big data technologies ndash Spark SQL Programming SPARK in Scala proficiency in SQL to write complex SQL queries Strong data analysis and troubleshooting skills Domain knowledge of Capital Market is plus Knowledge of shell scripts and other languages including Python, R, Java is plus At least 2 years of hands on experience on Big Data Engineering on AWS Minimum 1 year experience with Spark, Scala Experience with relational databases (Preferably Oracle) Good knowledge of Linux OS and Shell scripting Experience working on complex distributed information systems Experience with version control systems, preferable SVN, git. Strong work ethic in a mission-critical 24x7 diverse environment with multiple vendorcustomer relationships.

Keywords: Triumph Services, Rockville , Cloud/Data Engineer, Other , Rockville, Maryland