apache-iceberg-sink-connector

Data

Updated 12 hours ago

Apache-Iceberg-Sink-Connector Overview

Secure your stack with a hardened Apache Iceberg Sink Connector image freshly-built by Minimus. Minimus images always include the most up-to-date package version for all packages and dependencies contained in the image.

Use the Apache Iceberg Sink Connector image to run Kafka Connect with the Apache Iceberg sink connector plugin pre-installed, streaming records from Kafka topics directly into Apache Iceberg tables.

Try It Out

The Apache Iceberg Sink Connector image runs as a Kafka Connect worker, so it needs an Apache Kafka broker to connect to. The steps below start a single-node Kafka broker and the connector worker using Minimus images, then confirm the Iceberg sink plugin is loaded.

Step 1 - Create a Docker Network

Create a shared network so the Connect worker can reach the Kafka broker by container name:

docker network create iceberg-connect-net

Step 2 - Start a Kafka Broker

Start a single-node Kafka broker using the Minimus Kafka image:

docker run -d --name kafka \
  --net iceberg-connect-net \
  -e KAFKA_NODE_ID=1 \
  -e KAFKA_PROCESS_ROLES=broker,controller \
  -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093 \
  -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 \
  -e KAFKA_CONTROLLER_QUORUM_VOTERS=1@kafka:9093 \
  -e KAFKA_CONTROLLER_LISTENER_NAMES=CONTROLLER \
  -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT \
  -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
  -e CLUSTER_ID=MkU3OEVBNTcwNTJENDM2Qk \
  reg.mini.dev/kafka

Step 3 - Start the Connect Worker

Run the Apache Iceberg Sink Connector image as a Kafka Connect worker pointed at the broker:

docker run -d --name iceberg-connect \
  --net iceberg-connect-net \
  -p 8083:8083 \
  -e CONNECT_BOOTSTRAP_SERVERS=kafka:9092 \
  -e CONNECT_REST_ADVERTISED_HOST_NAME=iceberg-connect \
  -e CONNECT_GROUP_ID=iceberg-connect-cluster \
  -e CONNECT_CONFIG_STORAGE_TOPIC=_iceberg-connect-configs \
  -e CONNECT_OFFSET_STORAGE_TOPIC=_iceberg-connect-offsets \
  -e CONNECT_STATUS_STORAGE_TOPIC=_iceberg-connect-status \
  -e CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_STATUS_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_KEY_CONVERTER=org.apache.kafka.connect.json.JsonConverter \
  -e CONNECT_VALUE_CONVERTER=org.apache.kafka.connect.json.JsonConverter \
  reg.mini.dev/apache-iceberg-sink-connector

Step 4 - Confirm the Iceberg Sink Plugin Is Loaded

Once the worker is up, query the Kafka Connect REST API for the available connector plugins and confirm the Iceberg sink connector is present:

curl -s http://localhost:8083/connector-plugins | grep IcebergSinkConnector

{"class":"org.apache.iceberg.connect.IcebergSinkConnector", ...}

Step 5 - Clean Up

Stop and remove the containers and network:

docker rm -f iceberg-connect kafka
docker network rm iceberg-connect-net

Technical Considerations

The Apache Iceberg Sink Connector image provided by Minimus is a slim, security-hardened alternative to the public image from Docker Hub. The images are largely interchangeable, with a few differences as noted below.

Apache-Iceberg-Sink-Connector built by Minimus:

Runs as a non-root user by default.
Listens on port 8083/TCP by default but does not expose it.
Drill down on the version specification tab to see the default user, listening ports, entrypoint, volumes, environment variables, etc.

The Payoff

A hardened, minimal image that will remain more secure for the long run and accrue vulnerabilities at a slower rate.

See the risk reduction dashboard for a detailed CVE comparison over the past 30 days.
Review the compliance report to see the default hardening and security configurations for the image.

Terms & Info

Trademark

This catalog is published by Minimus. All product names, logos, and marks, other than those belonging to Minimus, shown are owned by their respective rights holders and appear here only to identify the open source software each image contains. Minimus claims no ownership of those marks and implies no affiliation with, endorsement by, certification by, or sponsorship by any rights holder.

Disclaimer

Images are provided "as-is" without warranty of any kind. "Hardened" refers to the security configuration applied at the time of build and does not constitute a guarantee of ongoing security or absence of vulnerabilities. The free tier is provided without support, SLA, or guaranteed patching timelines. Security updates may be applied to paid subscriptions before or instead of free tier images. By pulling or using any image you agree to our Terms of Use.