Time since we first saw it
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,766 similar postings that came down. Coming down is not always a hire.
Bengaluru, KA, India
Actively hiring, first seen by us 3 days agoAt ZenteiQ we're building the data foundation that powers next-generation Large Language Models.
You'll work closely with AI Researchers, ML Engineers and Platform Engineers to solve challenging data problems across web-scale datasets, unstructured data, multilingual corpora and benchmark datasets.
Source and ingest large datasets from Common Crawl and public web sources
Build distributed pipelines with Apache Spark, Ray or Dask
Design cleaning, filtering and quality pipelines for LLM pretraining corpora
Implement document-level and corpus-level deduplication
Identify and mitigate benchmark contamination in training corpora
At ZenteiQ we're building the data foundation that powers next-generation Large Language Models. As a Data Engineer on the corpus team, you'll build scalable pipelines to source, process, curate, filter and prepare large-scale corpora for training and evaluating foundation models.
You'll work closely with AI Researchers, ML Engineers and Platform Engineers to solve challenging data problems across web-scale datasets, unstructured data, multilingual corpora and benchmark datasets.
Source and ingest large-scale datasets from Common Crawl, public web sources, open-source repositories and other data sources.
Develop and optimise distributed data pipelines using tools such as Apache Spark, Ray and Dask.
Design data cleaning, filtering, quality assessment and curation pipelines for LLM pretraining corpora.
Implement document-level and corpus-level deduplication strategies.
Identify and mitigate benchmark contamination and evaluation-data leakage in training corpora.
ZenteiQ is a deep-tech company born out of IISc Bangalore, building Scientific Intelligence Infrastructure: physics-native AI for engineering, manufacturing, energy, mobility and national systems.
In the employer’s words · Read in context
Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,766 similar postings that came down. Coming down is not always a hire.
5 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- 2–5+ years of experience in Data Engineering, AI Data Engineering, ML Data Engineering or a related field. - Strong Python programming skills. - Experience working with large-scale unstructured datasets.
- Research-level understanding of synthetic data generation techniques for model training and evaluation. - Familiarity with Rust for high-performance data processing. - Experience with multilingual or domain-specific corpora.
- Experience with HTML parsing and web/document extraction libraries such as Trafilatura, BeautifulSoup, lxml or Resiliparse. - Familiarity with GCS/GCP and cloud-based data processing. - Experience working with large-scale formats such as JSONL, Parquet or Arrow.
- Prepare and curate benchmark and evaluation datasets. - Work with large-scale datasets stored and processed through GCS/GCP. - Monitor data quality, pipeline performance, storage and processing efficiency.
- Build scalable data processing pipelines for large unstructured corpora. - Develop and optimise distributed data pipelines using tools such as Apache Spark, Ray and Dask. - Extract and process content from web pages and documents using Trafilatura and related web/content extraction libraries.
- Hands-on experience with exact and near-duplicate detection and deduplication. - Familiarity with at least one distributed processing framework: Apache Spark, Ray or Dask. - Understanding of web-scale data ingestion, such as Common Crawl or other large public datasets.
- Experience building or maintaining LLM pretraining corpora. - Familiarity with Hugging Face Datasets, DataTrove or NVIDIA NeMo Curator. - Experience with WARC/WET/WAT and large-scale Common Crawl processing.
Complete your application on zenteiqai.keka.com. The employer’s form will show what is required.
Already applied? Track this application
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Bengaluru, KA, India
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.