public_git/bitnami-containers
1
0
Fork 0
mirror of https://github.com/bitnami/containers.git synced 2026-10-17 18:04:55 +02:00
Code Issues Projects Releases Wiki Activity

[bitnami/spark] Release 4.1.1-debian-12-r12 (#93021)

Browse Source 
Signed-off-by: Bitnami Bot <bitnami.bot@broadcom.com>
This commit is contained in:
Bitnami Bot
2026-05-09 23:54:33 +02:00
committed by GitHub
parent 94357ad369
commit 173c4e1bce
2 changed files with 12 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bitnami/spark/4.1/debian-12/Dockerfile
+4 -4
View File
@@ -8,7 +8,7 @@ ARG JAVA_EXTRA_SECURITY_DIR="/bitnami/java/extra-security"
ARG TARGETARCH
LABEL org.opencontainers.image.base.name="docker.io/bitnami/minideb:bookworm" \
org.opencontainers.image.created="2026-04-09T19:57:39Z" \
org.opencontainers.image.created="2026-05-09T21:04:42Z" \
org.opencontainers.image.description="Application packaged by Broadcom, Inc." \
org.opencontainers.image.documentation="https://github.com/bitnami/containers/tree/main/bitnami/spark/README.md" \
org.opencontainers.image.source="https://github.com/bitnami/containers/tree/main/bitnami/spark" \
@@ -31,8 +31,8 @@ RUN --mount=type=secret,id=downloads_url,env=SECRET_DOWNLOADS_URL \
mkdir -p /tmp/bitnami/pkg/cache/ ; cd /tmp/bitnami/pkg/cache/ || exit 1 ; \
COMPONENTS=( \
"nss-wrapper-1.1.16-0-linux-${OS_ARCH}-debian-12" \
"python-3.12.13-4-linux-${OS_ARCH}-debian-12" \
"jre-17.0.18-10-1-linux-${OS_ARCH}-debian-12" \
"python-3.12.13-9-linux-${OS_ARCH}-debian-12" \
"jre-17.0.19-11-1-linux-${OS_ARCH}-debian-12" \
"spark-4.1.1-4-linux-${OS_ARCH}-debian-12" \
) ; \
for COMPONENT in "${COMPONENTS[@]}"; do \
@@ -56,7 +56,7 @@ RUN /opt/bitnami/scripts/spark/postunpack.sh
RUN /opt/bitnami/scripts/java/postunpack.sh
ENV APP_VERSION="4.1.1" \
BITNAMI_APP_NAME="spark" \
IMAGE_REVISION="11" \
IMAGE_REVISION="12" \
JAVA_HOME="/opt/bitnami/java" \
LD_LIBRARY_PATH="/opt/bitnami/python/lib:/opt/bitnami/spark/venv/lib/python3.12/site-packages/numpy.libs:$LD_LIBRARY_PATH" \
LIBNSS_WRAPPER_PATH="/opt/bitnami/common/lib/libnss_wrapper.so" \
bitnami/spark/README.md
+8 -89
View File
@@ -7,13 +7,17 @@ Trademarks: This software listing is packaged by Bitnami. The respective tradema
## TL;DR
### Docker Compose
```console
docker run --name spark bitnami/spark:latest
```
You can find the available configuration options in the [Environment Variables](#environment-variables) section.
## Using `docker-compose.yml`
The docker-compose.yaml file of this container can be found in the [Bitnami Containers repository](https://github.com/bitnami/containers/).
[https://github.com/bitnami/containers/tree/main/bitnami/spark/docker-compose.yml](https://github.com/bitnami/containers/tree/main/bitnami/spark/docker-compose.yml)
Please be aware this file has not undergone internal testing. Consequently, we advise its use exclusively for development or testing purposes. For production-ready deployments, we highly recommend utilizing its associated [Bitnami Helm chart](https://github.com/bitnami/charts/tree/main/bitnami/spark).
## Why use Bitnami Secure Images?
@@ -48,10 +52,6 @@ Learn more about the Bitnami tagging policy and the difference between rolling t
The Bitnami Apache Spark Docker image is only available to [Bitnami Secure Images](https://bitnami.com) customers.
## Using `docker-compose.yaml`
Please be aware this file has not undergone internal testing. Consequently, we advise its use exclusively for development or testing purposes. For production-ready deployments, we highly recommend utilizing its associated [Bitnami Helm chart](https://github.com/bitnami/charts/tree/main/bitnami/spark).
## Configuration
The following sections describe environment variables and related settings.
@@ -103,8 +103,6 @@ Additionally, more environment variables natively supported by Apache Spark can
For example, you could still use `SPARK_WORKER_CORES` or `SPARK_WORKER_MEMORY` to configure the number of cores and the amount of memory to be used by a worker machine.
When you start the spark image, you can adjust the configuration of the instance by passing one or more environment variables either on the docker-compose file or on the `docker run` command line.
### Security
The Bitnami Apache Spark docker image supports enabling RPC authentication, RPC encryption and local storage encryption easily using the following environment variables in all the nodes of the cluster.
@@ -136,18 +134,6 @@ Additionally, SSL configuration can be easily activated following the next steps
2. You need to mount your spark `keystore` and `truststore` files to `/opt/bitnami/spark/conf/certs`. Please note they should be called `spark-keystore.jks` and `spark-truststore.jks` and they should be in JKS format.
### Setting up an Apache Spark cluster
A Apache Spark cluster can easily be set up with the default docker-compose.yml file from the root of this repository. The docker-compose includes two different services, `spark-master` and `spark-worker.`
By default, when you deploy the docker-compose file you will get an Apache Spark cluster with 1 master and 1 worker.
If you want N workers, all you need to do is start the docker-compose deployment with the following command:
```console
docker-compose up --scale spark-worker=3
```
### Mount a custom configuration file
The image looks for configuration in the `conf/` directory of `/opt/bitnami/spark`.
@@ -197,74 +183,7 @@ The Bitnami Apache Spark Docker image from the [Bitnami Secure Images](https://g
## Logging
The Bitnami Apache Spark Docker image sends the container logs to the `stdout`. To view the logs:
```console
docker logs spark
```
or using Docker Compose:
```console
docker-compose logs spark
```
You can configure the containers [logging driver](https://docs.docker.com/engine/admin/logging/overview/) using the `--log-driver` option if you wish to consume the container logs differently. In the default configuration docker uses the `json-file` driver.
## Maintenance
The following sections describe how to back up, restore, and upgrade the image.
### Backing up your container
To backup your data, configuration and logs, follow these simple steps:
1. Stop the currently running container
```console
docker stop spark
```
or using Docker Compose:
```console
docker-compose stop spark
```
2. Run the backup command
We need to mount two volumes in a container we will use to create the backup: a directory on your host to store the backup in, and the volumes from the container we just stopped so we can access the data.
```console
docker run --rm -v /path/to/spark-backups:/backups --volumes-from spark busybox \
cp -a /bitnami/spark /backups/latest
```
or using Docker Compose:
```console
docker run --rm -v /path/to/spark-backups:/backups --volumes-from `docker-compose ps -q spark` busybox \
cp -a /bitnami/spark /backups/latest
```
### Restoring your backup
To restore your backup, mount the backup as volumes in the container.
```console
docker run -v /path/to/spark-backups/latest:/bitnami/spark bitnami/spark:latest
```
or by modifying the [`docker-compose.yml`](https://github.com/bitnami/containers/blob/main/bitnami/spark/docker-compose.yml) file present in this repository:
```yaml
services:
spark:
...
volumes:
- /path/to/spark-backups/latest:/bitnami/spark
...
```
The Bitnami Apache Spark Docker image sends the container logs to the `stdout`. You can configure the containers [logging driver](https://docs.docker.com/engine/admin/logging/overview/) using the `--log-driver` option if you wish to consume the container logs differently. In the default configuration docker uses the `json-file` driver.
## Notable changes