Sydata Consulting, Inc. has the following job opening :
Job Title : Database Engineer
Job Location :1505 S.Pavilion Center Dr, Las Vegas ,NV 89135
Job Description :
- Creating a high-level design approach to build a data lake which will embrace the existing historical data and streaming data to suffice the need to process for Business.
- Design and development of Data transformation framework components to support BI process that gets the Single Complete Actionable View of customer sessions data.
- Extracting customer sessions data from various data sources into DBFS. This includes data from RabbitMQ using Flume service into HDFS, leverage Sqoop to load the transformed structured data to SQL server database into Teradata.
- Write, update spark programs to receive and publish the messages from rabbit to DBFS and PySpark to transform the structured and unstructured data to Machine Learning team for predictions.
- Performance tuning of jobs by monitoring long running jobs and enhancing them by necessary spark-streaming configuration properties.
- Resolving bugs/defects in Hadoop application by coordinating with team members of the project to assure a positive outcome.
- Automation of data pipelines using shell scripts in Linux.
Min Education :Bachelor Degree in Computer Science, Software Engineering or Information Systems or related or combination of education/training and/or progressive experience.
Apply by Mail :HR, 6494 Weathers Place, Suite # 100, San Diego, CA – 92121