Job Overview:
As a Data Engineer, you will be responsible for designing, developing, and maintaining the data architecture, data management systems, and data processing pipelines. You'll play a key role in managing and optimizing data models, creating robust data solutions to support the company's analytic and business intelligence needs.
Responsibilities:
Data Architecture: Design, create, and maintain robust and scalable data infrastructure, including data pipelines and data warehouses.
Data Ingestion & Processing: Develop and maintain ETL processes for large-scale data ingestion and transformation.
Database Management: Manage data distribution, storage, and retrieval activities in the databases. Ensure data quality and integrity.
Performance Tuning: Monitor data systems performance, recommending and implementing improvements and optimizations as necessary.
Collaboration: Collaborate with data scientists, BI analysts, and other stakeholders to understand data needs and deliver appropriate solutions.
Data Security: Implement measures to ensure data privacy and security in compliance with regulatory requirements.
Documentation: Document structures and types of business data for use in data projects and initiatives.
Requirements:
Education: Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field.
Experience:
At least 3-5 years of experience in a similar role with a focus on data warehousing and ETL processes.
Technical Skills: Proficient in SQL and experience with a variety of databases (both SQL and NoSQL). Experience with big data tools like Hadoop, Spark, or similar. Familiarity with data pipeline and workflow management tools like Airflow, NiFi, etc.
Soft Skills: Strong problem-solving skills, attention to detail, and ability to work collaboratively within a team.
Other: Familiarity with cloud services (AWS, Google Cloud, Azure) is a plus.
Disclaimer:
This job description is not an exhaustive list of the skills, efforts, duties, and responsibilities associated with the position. The company is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations, and ordinances.
Note: The actual duties and responsibilities may vary depending on the department's needs and the evolution of the role.