Back to jobs

Principal Engineering Lead — Content & Data Pipeline

Pay
USD 140,000–200,733.33/yearAnnual period assumed — pay source
Salary Range: 140,000.00 USD to 200,733.33 USD

#LI-CW1
Read the full posting
Work setting
Not stated
Employment
Full-time — employment source
Employment type in the employer’s posting: Full time
From the employer’s posting

What you’ll work on

Full posting

This role leads Wiley's Content & Data Pipeline function — the platform that takes scholarly content (journals, articles, books) from a wide mix of sources through ingestion, processing, and enrichment, and delivers it to internal AI systems, RAG pipelines, and external customers under the right entitlements.

The role sits alongside an existing dev lead who runs the current content ingestion and delivery pipeline; you take end-to-end ownership above that, expanding the platform to cover new source types, new delivery surfaces (especially internal AI / RAG and licensed external delivery), and higher standards of quality, freshness, and observability.

  • Own end-to-end architecture from vendor ingestion through enrichment to delivery

  • Run the pipeline in production, including corrections, retractions and version updates

  • Set standards for reliability, data quality, freshness monitoring and CI/CD

  • Manage and coach engineers on ingestion, processing and delivery

  • Own vendor and licensing-partner technical integrations and data contracts

From the employer’s posting
This role leads Wiley's Content & Data Pipeline function — the platform that takes scholarly content (journals, articles, books) from a wide mix of sources through ingestion, processing, and enrichment, and delivers it to internal AI systems, RAG pipelines, and external customers under the right entitlements. It is a hybrid role: you set technical direction and stay hands-on with architecture and code where it matters, and you also manage and grow a team of engineers who own the day-to-day pipeline work.
The role sits alongside an existing dev lead who runs the current content ingestion and delivery pipeline; you take end-to-end ownership above that, expanding the platform to cover new source types, new delivery surfaces (especially internal AI / RAG and licensed external delivery), and higher standards of quality, freshness, and observability.
Own end-to-end architecture for the content and data pipeline
Run and operate the pipeline in production and manage the content lifecycle — corrections, retractions, version updates
Set standards for pipeline reliability, data quality, freshness monitoring, testing, observability, and CI/CD across the team's services.
Manage and coach a team of engineers spanning content ingestion, processing, and delivery.
Own vendor and licensing-partner relationships for content sources and delivery pipes — technical integration, data contracts, escalation, renewal input.

Before you apply

Employer’s visa record
22 H-1B petitions approved in fiscal 2024 and 2025For the employer as a whole. This posting doesn’t say whether the role is sponsored. US Citizenship and Immigration Services.
Apply at John Wiley and Sons

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 15, 202637 days ago
Recorded sightings
38
Last seen by us
Nov 15, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Pay

In line with similar jobs

$140K–$201K against $155K–$227K, the middle half of what similar US postings state.

From 3,272 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 37

Similar jobs usually come down between day 8 and day 57.

At John Wiley and Sons, jobs usually come down by about day 33.

From 9,995 similar postings that came down, and 13 of John Wiley and Sons’s. Coming down is not always a hire.

John Wiley and Sons is hiring

9 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Hiring at a steady pace
  • Takes jobs down quickly
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • AWS
  • BigQuery
  • DynamoDB
  • GCP
  • Kafka
  • S3
  • Airflow
  • Dagster
Where the posting names them
- AWS at production depth — S3, Lambda, ECS or EKS, SQS / SNS, EventBridge, Step Functions, RDS, DynamoDB. Real IAM and cost fluency. - Python and SQL at production depth. - Workflow orchestration in production — Airflow, Step Functions, Dagster, or equivalent — with real experience in idempotency, backfills, retries, dependency management.
- Deep production experience designing and operating content or data pipelines end-to-end — ingestion, processing, delivery — at scale. - AWS at production depth — S3, Lambda, ECS or EKS, SQS / SNS, EventBridge, Step Functions, RDS, DynamoDB. Real IAM and cost fluency. - Python and SQL at production depth.
- Scholarly publishing specifics — JATS / BITS XML, DOI infrastructure, Crossref deposits, ORCID / ROR identity, MeSH and similar controlled vocabularies. - BigQuery / GCP experience — useful for the secondary consumer surface, especially as our AI teams standardise there. - Specialised publishing tooling — XSLT at scale, existing content-management systems, or custom XML processing frameworks.
- Specialised publishing tooling — XSLT at scale, existing content-management systems, or custom XML processing frameworks. - Streaming ingestion — Kafka, Kinesis, Pub/Sub — for near-real-time content updates. - Vector search and embedding pipelines — useful for the internal AI / RAG delivery surface.
- Python and SQL at production depth. - Workflow orchestration in production — Airflow, Step Functions, Dagster, or equivalent — with real experience in idempotency, backfills, retries, dependency management. - Content transformation experience — XML processing, schema-driven transformation, structured content formats.

Your next step

Complete your application on wiley.wd1.myworkdayjobs.com. The employer’s form will show what is required.

Applying through Workday

  • You create an account with each employer before you can upload anything. Every Workday employer has its own account, so one sign-up doesn’t carry over.
  • “Autofill with Resume” comes right after sign-in, before the My Information and My Experience steps. Check every field it fills in those steps, since that’s what recruiters see.
  • Your application status lives in that employer’s Candidate Home, under the same sign-in.

Already applied? Track this application

Job description

View original posting ↗

We believe in bold ideas, diverse perspectives, and the drive to transform knowledge into impact. Here, your curiosity fuels progress, your voice shapes innovation, and your ambition helps redefine what’s possible within science and learning. We are a culture that obsesses over impact, challenges, and drives what’s next to power infinite possibilities for our customers, colleagues and society at large.

About the Role:

About the role 

This role leads Wiley's Content & Data Pipeline function — the platform that takes scholarly content (journals, articles, books) from a wide mix of sources through ingestion, processing, and enrichment, and delivers it to internal AI systems, RAG pipelines, and external customers under the right entitlements. It is a hybrid role: you set technical direction and stay hands-on with architecture and code where it matters, and you also manage and grow a team of engineers who own the day-to-day pipeline work. 

The role sits alongside an existing dev lead who runs the current content ingestion and delivery pipeline; you take end-to-end ownership above that, expanding the platform to cover new source types, new delivery surfaces (especially internal AI / RAG and licensed external delivery), and higher standards of quality, freshness, and observability. 

The mandate spans the full content lifecycle — ingestion, transformation, enrichment, metadata management, rights-aware delivery, and vendor / supplier integration — across scholarly publications and books. Stakeholders include product, business, licensing partners, and vendors, and part of the job is turning their needs into a coherent, prioritized platform roadmap. 

 

We are looking for someone who leads by doing and by coaching — who codes, reviews, unblocks, and mentors, while also owning delivery, and stakeholder alignment for the team. 

 

What you'll do 

Technical direction and architecture 

  • Own end-to-end architecture for the content and data pipeline: ingestion from vendors and public sources, transformation and enrichment, metadata management, and delivery to internal AI / RAG systems and external customers. 
  • Run and operate the pipeline in production and manage the content lifecycle — corrections, retractions, version updates, metadata deposits and registry syndication, and rights and entitlements enforced at every delivery channels. 
  • Set standards for pipeline reliability, data quality, freshness monitoring, testing, observability, and CI/CD across the team's services. 
  • Codify architectural decisions in ADRs; drive convergence on shared patterns across ingestion, processing, and delivery. 
  • Stay hands-on — write code, run architecture reviews, prototype, and unblock the team where needed. 

Team leadership and delivery 

  • Manage and coach a team of engineers spanning content ingestion, processing, and delivery.  
  • Partner closely with the existing dev lead on the current pipeline; align on ownership boundaries and coordinate roadmap execution. 
  • Own delivery commitments — scope, sequencing, dependencies, and honest tradeoff conversations with stakeholders when reality does not match the plan. 
  • Run engineering practices that scale: code review discipline, on-call rotation, incident response and postmortems, quality gates in CI. 

Stakeholder partnership 

  • Partner with product and business teams on capability priorities — what content the platform ingests next, what delivery surfaces get built, what quality standards matter. 
  • Own vendor and licensing-partner relationships for content sources and delivery pipes — technical integration, data contracts, escalation, renewal input. 
  • Translate stakeholder needs (editorial ops, product, licensing) into a coherent platform roadmap; push back honestly when requests do not fit the platform's direction. 
  • Communicate technical strategy and tradeoffs upward and outward — not just to peer engineering leaders but to product, business, and executive audiences. 

 

What you'll own 

  • The Content & Data Pipeline platform end-to-end — ingestion through delivery — including architecture, standards, delivery SLAs, and data quality guarantees to internal and external consumers. 
  • The Content & Data Pipeline team — build and delivery 
  • Vendor and licensing-partner technical relationships for content sources and delivery pipes in scope. 
  • The technical roadmap for expanding the platform to new sources, new delivery surfaces, and higher engineering bars, codified in ADRs and communicated to stakeholders. 

 

Must-have skills 

Leadership 

  • Player-coach engineering leadership at scale — you have led an engineering team while remaining genuinely hands-on with architecture and code.  
  • Cross-team influence with product, business, vendor, and executive stakeholders. Able to communicate technical strategy in language each audience actually uses. 
  • Vendor and partner relationship management — technical integration, contract execution, renewal input, escalation. 
  • Honest tradeoff communication — you make scope, quality, and timeline conversations concrete rather than optimistic. 

Technical depth 

  • Deep production experience designing and operating content or data pipelines end-to-end — ingestion, processing, delivery — at scale. 
  • AWS at production depth — S3, Lambda, ECS or EKS, SQS / SNS, EventBridge, Step Functions, RDS, DynamoDB. Real IAM and cost fluency. 
  • Python and SQL at production depth. 
  • Workflow orchestration in production — Airflow, Step Functions, Dagster, or equivalent — with real experience in idempotency, backfills, retries, dependency management. 
  • Content transformation experience — XML processing, schema-driven transformation, structured content formats. 
  • Data quality practice: schema validation, defensive parsing, incremental correctness checks, freshness monitoring, alerting on drift. 
  • Distributed systems literacy: idempotency, retries, duplicate handling, partial failure, replay. 
  • Real production experience with API-based and bulk ingestion from external sources — auth, pagination, rate limiting, silent schema changes. 
  • Testing and CI/CD discipline for pipelines — unit, integration, and data-quality tests in CI; safe deployment patterns. 
  • Observability practice — logs, metrics, lineage, and the discipline of knowing whether the pipeline is actually healthy vs. just running. 
  • Active use of AI coding agents in your day-to-day workflow — always with review. 

 

Hugely preferred: content pipeline experience 

Scholarly publishing — journals, articles, books — is our domain and our first preference. But the core skill is building pipelines for content, not just data, and that experience carries over from any content-heavy environment. 

  • You have built or run pipelines that ingest, process, and deliver content in structured formats — XML, HTML, EPUB, PDF — where structure and metadata matter. 
  • You have dealt with content that gets corrected, updated, or pulled after it ships, and built pipelines that handle that cleanly. 
  • You have delivered the same content to different consumers in different shapes — feeds, APIs, search, AI systems. 
  • You have worked with content that has usage rights or access restrictions attached. 
  • You have partnered with editorial, production, or content-operations teams and earned their trust. 

Scholarly-specific knowledge — JATS, DOIs, Crossref, retraction handling — is a strong plus. We can teach the acronyms to someone who already has the pipeline instincts. 

 

Nice to have 

  • Scholarly publishing specifics — JATS / BITS XML, DOI infrastructure, Crossref deposits, ORCID / ROR identity, MeSH and similar controlled vocabularies. 
  • BigQuery / GCP experience — useful for the secondary consumer surface, especially as our AI teams standardise there. 
  • Specialised publishing tooling — XSLT at scale, existing content-management systems, or custom XML processing frameworks. 
  • Streaming ingestion — Kafka, Kinesis, Pub/Sub — for near-real-time content updates. 
  • Vector search and embedding pipelines — useful for the internal AI / RAG delivery surface. 
  • Prior partnership with ML, AI, or agent teams as a downstream consumer of your platform. 
  • Data catalog and lineage tooling: DataHub, Amundsen, or equivalent. 
  • Existing relationships in the scholarly publishing ecosystem — vendors, standards bodies, or peer engineering leaders at similar organisations. 

 

How we work 

  • Player-coach leadership. We lead by doing and by coaching. Purely delegating leaders and purely heads-down principal engineers are both mismatches for this role. 
  • Content as a first-class engineering concern. Scholarly content has production standards, versioning semantics, and rights implications that generic data pipelines do not — and we engineer to them. 
  • End-to-end ownership. We own ingestion through delivery. We do not throw work over the wall between ingestion, processing, and delivery teams; we design across them. 
  • Stakeholder partnership. Product, business, vendors, and executives are partners, not audiences. We share evidence and tradeoffs openly, and we push back honestly when needed. 
  • Learning agility. The scholarly content and AI landscapes both move quickly. We hire leaders who show up curious about new territory rather than defensive of what they already know. 
  • Quality is a first-class engineering concern. Data-quality bugs are engineering bugs, not “the upstream source's problem.” 
  • Agentic engineering is a strong expectation, including at leadership levels. We use AI coding agents actively — always with review — and we set the standards our teams follow. 
  • Production ownership. Engineers, including engineering leaders, own the platforms they build — including on-call, incident response, and using observability tools to diagnose their own systems. 

We power infinite possibilities.


For more than 200 years, we've transformed knowledge into discoveries that shape the world. Today, our global team of innovators, creators, and experts is driving what's next in science, education, and publishing—creating impact that reaches everywhere. 

 

We're not just observers of progress. We're the ones accelerating scientific breakthroughs, advancing learning, and sparking innovation that redefines entire fields and improves lives. 


Here, your talent matters. Your ideas have room to grow. And your work creates breakthroughs that can change everything. 

Wiley is an equal opportunity/affirmative action employer. We evaluate all qualified applicants and treat all qualified applicants and employees without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, disability, protected veteran status, genetic information, or based on any individual's status in any group or class protected by applicable federal, state or local laws. Wiley is also committed to providing reasonable accommodation to applicants and employees with disabilities. Applicants who require accommodation to participate in the job application process may contact tasupport@wiley.com for assistance.


We are proud that our workplace promotes continual learning and internal mobility. We offer meeting-free Friday afternoons allowing more time for heads down work and professional development, and through a robust body of employee programing we facilitate a wide range of opportunities to foster community, learn, and grow.

We are committed to fair, transparent pay, and we strive to provide competitive compensation in addition to a comprehensive benefits package. The range below represents Wiley's good faith and reasonable estimate of the base pay for this role at the time of posting roles in the United Kingdom, Canada, USA, Austria, Czechia, Denmark, France, Greece, Italy, Netherlands, Romania, or Spain. It is anticipated that most qualified candidates will fall within the range, however the ultimate salary offered for this role may be higher or lower and will be set based on a variety of non-discriminatory factors, including but not limited to, geographic location, skills, and competencies.

When applying, please attach your resume/CV to be considered.

Salary Range:

140,000.00 USD to 200,733.33 USD

#LI-CW1

About John Wiley and Sons

People
About 4,500 in 2026, down 13% from the year before.Annual report (SEC Form 10-K)
Company
Listed as WLY.

Demand for Data Leadership roles

New Data Leadership postings at the employers we follow are up 8% over the last 6 weeks compared with the 6 before.

Weekly new Data Leadership postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 751, the most 1,165, and the latest week had 775.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Leadership, by the work the posting describes.

Data leaders run data teams or design how a company's data fits together. Managers hire, set priorities and answer for whether the data work pays off; architects choose the systems and the shape of the data.

Expect years of hands-on data work, experience leading people or projects, and the ability to explain tradeoffs to executives.

Unlike a senior engineer or analyst, most of a data leader's week goes to people, planning and decisions rather than building.

US government figures for computer and information systems managers

Median yearly pay, May 2025
$175,140
Expected change, 2025 to 2035
15.8% more jobsagainst 3.5% for all jobs

The government has no code for data leaders, so these figures are for the closest group, computer and information systems managers, which includes data processing and data operations managers.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 11-3021.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Salary Range: 140,000.00 USD to 200,733.33 USD

#LI-CW1
Location & working pattern

Hoboken (HQ), NJ, USA

About the role This role leads Wiley's Content & Data Pipeline function — the platform that takes scholarly content (journals, articles, books) from a wide mix of sources through ingestion, processing, and enrichment, and delivers it to internal AI systems, RAG pipelines, and external customers under the right entitlements. It is a hybrid role: you set technical direction and stay hands-on with architecture and code where it matters, and you also manage and grow a team of engineers who own the day-to-day pipeline work. The role sits alongside an existing dev lead who runs the current content ingestion and delivery pipeline; you take end-to-end ownership above that, expanding the platform to cover new source types, new delivery surfaces (especially internal AI / RAG and licensed external delivery), and higher standards of quality, freshness, and observability.
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.