Ingress
Ingress is how data enters a workflow: uploads on File Input, HTTP triggers on REST Input, or catalog sync through Data Connector nodes backed by PyAirbyte. Parsed tabular data is stored securely in your workspace (see Storage and processing). Connector credentials are configured in node settings and resolved from Vault at run time, never stored in workflow config.
Upstream connector catalog
Section titled “Upstream connector catalog”Ingress mechanisms
Section titled “Ingress mechanisms”Ingest data from major databases, SaaS, files, APIs, streams, and custodian feeds into governed workflow datasets.
| Mechanism | Description |
|---|---|
| File Input | Upload or drag-and-drop. Tabular, FIX, SWIFT/MT940, and OFX/QFX files parse to row datasets on the workflow engine; PDF and office documents route through Docling on the platform cluster. See Supported formats for the full extension list. |
| REST Input | External systems POST JSON to trigger a run and populate the input node output port. |
| Data Connector (Airbyte) | Airbyte-catalog sources via PyAirbyte: Notion, Postgres, S3, and hundreds more. No separate Airbyte platform server. |
PyAirbyte execution path
Section titled “PyAirbyte execution path” Data Connector Workflow engine PyAirbyte sidecar Arrow file store
Docker/Java connectors spawn short-lived Kubernetes Jobs in the workspace namespace; Python/YAML connectors run in the sidecar.
- Python and YAML connectors run inside the workflow engine pod
- Docker and Java connectors run in isolated short-lived jobs with scanned, pinned images
- Connectors are baked into release images at build time (no runtime package installs on production hosts)
Related documentation
Section titled “Related documentation”- Supported formats - File Input extensions, FIX, SWIFT, and office formats
- Storage and processing - where ingested data is stored and how runs persist
- Egress - export and downstream connectors
- Data - connector secrets and section overview