Design and implement a scalable and efficient data engineering platform
The integration of data from multiple sources: organized, disorganized, structured and/or unstructured, into a centralized data repository. Creating and evolving data models, schemas and repositories to efficiently store and process data, accomplishing this through well designed, implemented and maintained ETL pipelines.
Be responsible for the detailed cataloguing of all data assets within Columbus (from his/her area of responsibility), not only for own use, but also to assist data professionals as they analyze available data.
Data quality and governance will be ensured and enforced by designing and implementing data quality standards, as well as governance policies, holding him/her accountable for architectures and/or data systems that promote data reliability, scalability and quality.
Will ensure data security and compliance to privacy regulations by developing and implementing the required security protocols and access controls. Throughout all this he/she will continually optimize the data repository’s query and processing performance, ensuring quick and efficiently processed data.
Internally the engineer has direct interfaces and interaction with the Production and Distribution departments (client oriented), the Engineering department (Control and measuring interfaces, both client and provider oriented), Technical department (process optimization, client oriented), OPS Planning (scheduling aspects, client oriented), and other IT Teams (L2, L3 and L4 teams, main providers).
Externally the engineer would liaise with external system suppliers, process/data technology consultants, and application software suppliers. Cooperation with external stakeholders like customers and/or suppliers are also frequent, in order to integrate external data systems with Columbus Stainless systems
QUALIFICATION
A Bachelor's or Master’s degree in Computer Science, Data Science, or related domain of study
EXPERIENCE
Minumum of 2 years, to 6 years experince (years of experince will determine the appointment Grade)
RELEVANT SKILLS/COMPETENCIES
Minimum of 4 years Data modelling, Data integration (ETL), System architecture design and System optimization experience.
Minimum of 3 years Software design and engineering principles experience.
Minimum of 2 years Project management experience.
CLOSING DATE
06th April 2025
Requirements
KEY PERFORMANCE INDICATORS
Design and implement a scalable and efficient data engineering platform
The integration of data from multiple sources: organized, disorganized, structured and/or unstructured, into a centralized data repository. Creating and evolving data models, schemas and repositories to efficiently store and process data, accomplishing this through well designed, implemented and maintained ETL pipelines.
Be responsible for the detailed cataloguing of all data assets within Columbus (from his/her area of responsibility), not only for own use, but also to assist data professionals as they analyze available data.
Data quality and governance will be ensured and enforced by designing and implementing data quality standards, as well as governance policies, holding him/her accountable for architectures and/or data systems that promote data reliability, scalability and quality.
Will ensure data security and compliance to privacy regulations by developing and implementing the required security protocols and access controls. Throughout all this he/she will continually optimize the data repository’s query and processing performance, ensuring quick and efficiently processed data.
Internally the engineer has direct interfaces and interaction with the Production and Distribution departments (client oriented), the Engineering department (Control and measuring interfaces, both client and provider oriented), Technical department (process optimization, client oriented), OPS Planning (scheduling aspects, client oriented), and other IT Teams (L2, L3 and L4 teams, main providers).
Externally the engineer would liaise with external system suppliers, process/data technology consultants, and application software suppliers. Cooperation with external stakeholders like customers and/or suppliers are also frequent, in order to integrate external data systems with Columbus Stainless systems
QUALIFICATION
A Bachelor's or Master’s degree in Computer Science, Data Science, or related domain of study
EXPERIENCE
Minumum of 2 years, to 6 years experince (years of experince will determine the appointment Grade)
RELEVANT SKILLS/COMPETENCIES
Minimum of 4 years Data modelling, Data integration (ETL), System architecture design and System optimization experience.
Minimum of 3 years Software design and engineering principles experience.
Minimum of 2 years Project management experience.
CLOSING DATE
06th April 2025
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.