Skip to content

The Pipeline FrameworkFast Java Function Flows on Quarkus

Write small typed business functions; TPF generates, validates, and reliably runs the Quarkus runtime around them

The Pipeline Framework
Async pipeline motion: an await step hands work through Kafka to an external provider while TPF persists payment records automatically.
Fastest path: design with Canvas
Use Canvas when you want to sketch the flow visually and download a runnable Quarkus scaffold. The same model can then be refined through YAML, Java functions, existing-method operators, type mappers, and runtime configuration.
Platform mode and transport mode are different decisions
`COMPUTE` and `FUNCTION` are platform modes: they decide whether TPF generates a standard Quarkus service runtime for containers and Kubernetes or a function-style runtime. REST, gRPC, and local are transport modes: they decide how generated components call each other.

Latest Articles

View feed

Recent DEV Community posts tagged tpf.

Loading articles...

Latest Releases

View all

Fresh from GitHub releases for The Pipeline Framework.

Loading releases...

Why Teams Choose TPF

What TPF Helps You Build

TPF pipelines are fast, forward-only flows of Java functions. They are not CI/CD job chains or coarse batch jobs. A step is one function in the flow: it receives a typed Java input, produces a typed Java output, and leaves the repeated Quarkus code to TPF.

What you write

Focused Java business functions such as validating a payment, enriching a record, parsing a document, or calling an existing Java method.

What TPF generates

REST, gRPC, local, and function-style code that calls those functions, plus runtime files that tell the generated runtime which function comes next.

What TPF runs

The generated runner that starts the flow, calls each step, tracks progress when configured, retries failed work, and routes failures.

How It Works

Write functions, not glue

You write typed Java functions and declare their order in YAML. TPF checks the flow at build time, including whether each function shape is valid, whether outputs match the next inputs, whether mappers convert types correctly, whether operators are valid, and whether the generated code can call each function cleanly.

A mapper translates between your domain types and transport or external-system types. An operator is an existing Java method or remote endpoint reused as a pipeline function.

Start with pipeline compilation when you want the build-time generation model, or code a step when you want to implement the function itself.

Generate the repeated Quarkus code

TPF creates the REST resources, gRPC services, local clients, function-style handlers, and runtime files that would otherwise become handwritten service glue. An adapter is the generated code around your business function: it lets another component call the function through the selected transport.

Generated code keeps business logic independent of whether a caller uses REST, gRPC, local in-process calls, or a serverless function entry point. That same generated runtime can be packaged as a standard Quarkus service for containers and Kubernetes or as a FUNCTION deployment when that model fits better. See Portable Serverless Functions and runtime layouts when you want the details behind platform mode, transport mode, logical placement, and Maven/container packaging.

Run the flow reliably

The generated runtime starts the flow, invokes each step, records execution state when configured, retries failed transitions, and exposes status/result endpoints for background work. For crash-surviving background execution, TPF can store accepted work outside the current JVM, recover it after restart, and send terminal execution failures to a DLQ, a dead-letter channel for investigation or replay.

The exact config value for this mode is QUEUE_ASYNC; start with the Orchestrator Runtime guide before using it in production.

Persistence and caching matter here more than the word "plugin" suggests. Persistence stores business outputs developers can query later from APIs, reports, or UIs. Caching protects expensive steps, speeds recomputation, and supports replay or rewind scenarios when downstream logic changes. Together they reduce the need to invent separate state stores, replay workflows, or read-model plumbing.

High availability is a runtime responsibility
TPF does not just generate code and walk away. In the background execution path, it owns stored execution state, dispatch, retry, crash recovery, and terminal failure publication. Your team still owns provider choice, duplicate-protection policy, retry budgets, observability thresholds, and deployment rollout.

Add cross-cutting work without hiding it

Plugins add declared cross-cutting work such as persistence, cache, telemetry, and logging. An aspect is the rule that says where the plugin runs, for example before or after a step; the plugin provides the implementation.

That keeps business functions focused on domain behaviour while TPF keeps generated plugin calls aligned across REST, gRPC, and local execution.

See State, Replay, and Queryable Data for the full persistence-and-caching story.

Operators preserve existing investment
If you already have proven Java libraries or remote endpoints, operators let you reuse them as pipeline functions. TPF still validates the method reference, input/output types, type-translation compatibility, and generated function call instead of turning reuse into hidden service glue.

What This Looks Like in Practice

Payments

Validate and enrich records, produce status output, reject one malformed item, and continue the workload.

Search

Crawl, parse, tokenize, index, cache intermediate results, and replay safely when source content changes.

Checkout and TPFGo

Pass stable checkpoints from one pipeline to the next without inventing custom handoff code.

AI enrichment

Reuse embedding, vector-search, or LLM helper libraries as operators inside a typed Java flow.

Serverless deployments

Run the same typed flow behind AWS Lambda, Azure Functions, or Google Cloud Run functions without rewriting the business code.

Container platforms

Run the same typed flow as a standard Quarkus service in containers or Kubernetes when you want service-style deployment and operations.

Start Here