Connector Catalog
Find source and sink plugins and the configuration fields that each accepts
The catalog lists source and sink plugins from the Apache Iggy connectors runtime. LaserData builds and signs the Rust plugins. Warden delivers them to deployment nodes. Their data path uses no JVM or garbage collector.
The tables list each plugin's key, purpose, and plugin_config fields. API responses mask fields marked as secrets. Each instance also uses the shared pipeline configuration.
Use the deployment's catalog endpoint to find enabled plugins for your account. Use its schema endpoint for the fields, types, and defaults of the running version.
Sink Connectors
Sinks read Iggy streams and write to external systems. Most support batch_size, include_metadata, include_checksum, include_origin_timestamp, payload_format, max_retries, retry_delay, and timeout. The table lists plugin-specific configuration.
| Plugin | Key | What it does | Main settings |
|---|---|---|---|
| PostgreSQL | postgres | Writes messages to a PostgreSQL table | connection_string (secret), target_table, auto_create_table, max_connections, payload_format such as bytea |
| ClickHouse | clickhouse | Inserts messages into a ClickHouse table | url, database, username, password (secret), table, insert_format such as json_each_row, timeout_seconds |
| Amazon Redshift | redshift | Loads messages into Redshift through S3-staged Parquet files | connection_string (secret), target_table, s3_bucket, s3_prefix, s3_endpoint, aws_region, aws_access_key_id and aws_secret_access_key (secrets) or aws_iam_role, archive, payload_format such as varbyte |
| Amazon S3 | s3 | Writes messages to S3 or S3-compatible object storage as rotated files | bucket, prefix, region, path_template with {stream}, {topic}, {date}, and {hour} placeholders, file_rotation, max_file_size, output_format such as json_lines, include_headers, max_attempts |
| Elasticsearch | elasticsearch | Indexes messages into an Elasticsearch index | url, index, create_index_if_not_exists, timeout_seconds, max_retries, retry_delay, retry_max_delay |
| RabbitMQ | rabbitmq | Publishes messages to RabbitMQ exchanges over AMQP | amqp_url (secret), exchange, exchange_type, routing_key, durable_exchange, delivery_mode, include_metadata, max_retries, retry_delay_secs, max_retry_delay_secs, timeout_secs |
| Quickwit | quickwit | Indexes messages into a Quickwit index | url, index |
| Meilisearch | meilisearch | Indexes messages as documents in Meilisearch | url, index, primary_key, document_action such as replace, create_index_if_not_exists, wait_for_tasks |
| MongoDB | mongodb | Writes messages to a MongoDB collection | connection_uri (secret), database, collection, auto_create_collection, payload_format such as binary |
| InfluxDB | influxdb | Writes messages as points to InfluxDB | version, url, org, bucket, token (secret), measurement, precision, include_stream_tag, include_topic_tag, include_partition_tag |
| SurrealDB | surrealdb | Writes messages to a SurrealDB table | endpoint, namespace, database, table, username, password (secret), auth_scope, use_tls, auto_define_table, define_indexes, include_headers, query_timeout, max_retry_delay |
| Apache Iceberg | iceberg | Writes messages to Iceberg tables through a catalog | tables, catalog_type such as rest, uri, warehouse, dynamic_routing with dynamic_route_field, and the object store settings store_url, store_class, store_region, store_path_style_access, store_access_key_id and store_secret_access_key (secrets) |
| Delta Lake | delta | Appends messages to a Delta Lake table on local, S3, Azure, or GCS storage | table_uri |
| Apache Doris | doris | Loads messages into Doris tables through Stream Load | fe_url, database, table, username, password (secret), label_prefix, timeout |
| HTTP | http_generic | POSTs message batches to an HTTP endpoint with retries | url, method, batch_mode such as ndjson, max_payload_size_bytes, success_status_codes, health_check_enabled and health_check_method, retry_backoff_multiplier, max_retry_delay, max_connections, tls_danger_accept_invalid_certs, and a headers table for values such as Authorization |
| Stdout | stdout | Prints messages to the runtime's standard output for debugging | print_payload |
The HTTP sink uses catalog key http_generic and runtime artifact name http.
Source Connectors
Sources bring data from external systems into an Iggy stream and topic. The runtime tracks progress and saves checkpoints, records of completed work, to resume after restart. See Source Checkpoints and Failover.
| Plugin | Key | What it does | Main settings |
|---|---|---|---|
| PostgreSQL | postgres | Polls tables and produces new rows as messages | connection_string (secret), mode such as polling, tables, tracking_column, initial_offset, poll_interval, batch_size, max_connections, snake_case_columns, include_metadata |
| Elasticsearch | elasticsearch | Polls an index for new documents by timestamp | url, index, timestamp_field, polling_interval, batch_size |
| HTTP | http_generic | Receives webhook POSTs on an embedded HTTP listener and produces the bodies as raw messages | listen_addr, admin_listen_addr, topic_path, auth_bearer_token (secret), management_token (secret), max_body_size_bytes, buffer_capacity, max_batch_size, include_http_metadata, forward_headers, endpoints with per-endpoint auth_type such as hmac-sha256 |
| InfluxDB | influxdb | Runs a Flux query on an interval and advances a cursor | version, url, org, token (secret), query with $cursor and $limit placeholders, cursor_field, initial_offset, poll_interval, batch_size, payload_format, include_metadata, max_retries, retry_delay, timeout |
| Random | random | Generates random messages for development and load testing | interval, max_count, messages_range, payload_size |
The HTTP source needs a reachable listener on the active source node. Cloud connector configuration does not open an ingress port or expose the admin listener.
Static endpoints are masked as one list because their URL IDs act as credentials. To change them, supply a complete replacement list.
Transforms
A transform changes message fields before a sink writes them or a source produces them. Instances can apply an ordered list of transforms:
| Transform | Purpose |
|---|---|
add_fields | Add fields with static or computed values |
delete_fields | Remove fields |
filter_fields | Keep only the listed fields |
update_fields | Change existing field values |
proto_convert | Convert to or from Protocol Buffers |
flat_buffer_convert | Convert to or from FlatBuffers |
avro_convert | Convert to or from Avro using a schema |
unwrap_envelope | Unwrap a nested payload envelope into the top-level message |
Transform configuration uses JSON values. The same structure applies when the surrounding plugin configuration uses JSON, YAML, or TOML.