Commit 5d87b9e7 authored by Pedro Eduardo Trujillo's avatar Pedro Eduardo Trujillo
Browse files

Merge branch 'dev' into 'master' 

Comprueba versión de Docker a nivel de server

See merge request redmic-project/docker/docker-deploy!64
parents 0fccc9de 54166b00

.env

+2 −2
Original line number Diff line number Diff line 
DOCKER_VERSION=24.0.7-cli-alpine3.18

OPENSSH_VERSION=9.3_p2-r0

DOCKER_VERSION=24.0.7-cli-alpine3.19

OPENSSH_VERSION=9.5_p1-r0

VERSION=unknown version

README.md

+112 −15
Original line number Diff line number Diff line 
# Docker deploy



Docker deployment utilities for REDMIC infrastructure.



You can use it to deploy your own services, supporting **Docker Compose** (both v1 and v2) and **Docker Swarm** environments.



## Actions



* **deploy**: Perform a service deployment at a Docker environment. Contains 3 stages:

  * *prepare-deploy*: Check dependencies, copy resources to deployment target host (*compose* files, service configurations...), prepare environment variables and directories, etc.

  * *do-deploy*: Launch service at deployment target host. Both standard (using `docker compose`) and *Swarm* (using `docker stack deploy`) modes are supported (deprecated versions too), but *Swarm* mode is recommended (even for single-node clusters).

  * *check-deploy*: Once deployment is done, this stage waits a defined time period for the service to being up and running (or stopped after run successfully). If service status remains stable after several checks, then it is considered successfully deployed.

* **create-nets**: Prepare deployment target host environment creating Docker networks which are external to service definition (not created by service deployment itself, defined as *external* in compose files).

* **relaunch**: Force a previously deployed service to update, relaunching it with the same service configuration. Available only for *Swarm* mode.

* **deploy**:



  Perform a service deployment at a Docker environment. Contains several stages:



  1. **definitions**:



     Set initial configuration values, getting environment values and with local defaults as fallback. Also prints the initial banner.



  1. **ssh-config**:



     Prepare connection to deployment target host environment. Set connection options, add identity and define functions to run commands at target.



  1. **check-env**:



     Check dependencies, version requirements and available modes at deployment target host environment.



  1. **check-mode**:



     Check which deploy mode (`Swarm` or `Compose`) will be used at deployment target host environment.



  1. **prepare-env**:



     Prepare environment variables, define directories to use and get names of services to deploy.



  1. **check-config**:



     Validate deployment configuration at *compose* files, using environment variables and deployment mode set.



  1. **prepare-deploy**:



     Prepare directories and copy resources to deployment target host (*compose* files, service configurations...).



  1. **do-deploy**:



     Launch service at deployment target host. Both standard (using `docker compose`) and *Swarm* (using `docker stack deploy`) modes are supported (deprecated versions too), but *Swarm* mode is recommended (even for single-node clusters).



  1. **check-deploy**:



     Once deployment is done, this stage waits a defined time period for the service to being up and running (or stopped after run successfully). If service status remains stable after several checks, then it is considered successfully deployed. This check can be ommited with `OMIT_STATUS_CHECK=1`.



* **create-nets**:



  Prepare deployment target host environment creating Docker networks which are external to service definition. A network is external when it's not created by service deployment itself, because is defined as *external* in compose files. Contains several stages:



  1. **definitions**:



     Set initial configuration values, getting environment values and with local defaults as fallback. Also prints the initial banner.



  1. **ssh-config**:



     Prepare connection to deployment target host environment. Set connection options, add identity and define functions to run commands at target.



  1. **check-env**:



     Check dependencies, version requirements and available modes at deployment target host environment.



  1. **do-create-nets**:



     Prepare networks creation command for supported mode and run it at deployment target host environment.



* **relaunch**:



  Force a previously deployed service to update, relaunching it with the same service configuration. Available only for *Swarm* mode. Contains several stages:



  1. **definitions**:



     Set initial configuration values, getting environment values and with local defaults as fallback. Also prints the initial banner.



  1. **ssh-config**:



     Prepare connection to deployment target host environment. Set connection options, add identity and define functions to run commands at target.



  1. **check-env**:



     Check dependencies, version requirements and available modes at deployment target host environment.



  1. **prepare-relaunch**:



     Obtain single service name to relaunch, because `SERVICE` value might be a prefix for several service names. Check if service exists at deployment target host environment.



  1. **prepare-registry**:



     Only when using `USE_IMAGE_DIGEST=1`. Perform login to registry when using credentials, to be able to get updated images.



  1. **prepare-digest**:



     Only when using `USE_IMAGE_DIGEST=1`. Before relaunching service, get updated image and its current digest data.



  1. **do-relaunch**:



     Run service relaunch at deployment target host environment.



## Usage
@@ -31,13 +117,19 @@ As you can see, configuration is possible through environment variables and by s 


Using environment variables, you can configure:



* Behaviour of this image itself.

* Deployment target host environment (where you are deploying to) for service deployment configuration and deployed service environment variables (the latter only when action is *deploy* and using the `ENV_PREFIX` prefix in your variable names).

* Behaviour of `docker-deploy` itself.

* Deployment target host (where you are deploying to) environment, for:

  * service deployment configuration (values used to configure the deployment), but not exposed into the deployed service.

  * deployed service environment variables, only when action is *deploy* and you are using the `ENV_PREFIX` (`DD_` by default) prefix in your variable names.



Using script parameters you can set:



* When action is *deploy*, deployment target host environment (where you are deploying to) for service deployment configuration and deployed service environment variables. These parameters overwrite previous environment values, including those defined using the `ENV_PREFIX` prefix.

* When action is *create-nets*, the name of external networks to create.

* When action is *deploy*:

  * deployment target host (where you are deploying to) environment, for:

    * service deployment configuration (values used to configure the deployment), but not exposed into the deployed service.

    * deployed service environment variables, using names without prefix. These parameters will overwrite previous environment values, including those defined using the `ENV_PREFIX` (`DD_` by default) prefix.

* When action is *create-nets*:

  * the name of external networks to create.



## Configuration
@@ -50,9 +142,10 @@ You may define these environment variables (**bold** are mandatory): 
| **DEPLOY_KEY** | - | Private key used to authenticate, paired with a public key accepted by remote host. |

| **SSH_REMOTE** | - | SSH user and hostname (DNS or IP) of remote host where you are going to deploy. |

| **STACK** | - | Name of Docker stack (*Swarm* mode) or project (*Compose* mode) used to wrap deployed services. |

| *ALLOW_COMPOSE_ENV_FILE_INTERPOLATION* | `0` | Allow passing variable values directly from `COMPOSE_ENV_FILE_NAME` file (`.env` by default), to let *Compose* interpolate variables used into values. By default, values will be single-quoted before checking config and deploying with *Compose*, to avoid getting unwanted variable resolution. Useful only for *Compose* mode. |

| *COMPOSE_ENV_FILE_NAME* | `.env` | Name of variable values definition file. |

| *COMPOSE_FILE* | `compose.yaml` | Name of service definition file. Multiple files are supported, separated by colon (`:`). |

| *DEFAULT_DEPLOY_FILES* | `*compose*.y*ml .env` | Files needed for deployment. Used only if `DEPLOY_DIR_NAME` directory does not exist. |

| *DEFAULT_DEPLOY_FILES* | `*compose*.y*ml ${COMPOSE_ENV_FILE_NAME}` | Files needed for deployment. Used only if `DEPLOY_DIR_NAME` directory does not exist. |

| *DEPLOY_DIR_NAME* | `deploy` | Name of directory containing files needed for deployment. If directory exists, `DEFAULT_DEPLOY_FILES` is ignored and all contents are copied to deployment target host. |

| *DEPLOY_PATH* | `~` | Path in deployment target host where deployment directory (used to hold temporary files) will be created. |

| *ENV_PREFIX* | `DD_` | Prefix used to identify variables to be defined in deployment target host environment and service, available there without this prefix. Change this if default value collides with the beginning of your variable names. |
@@ -61,6 +154,7 @@ You may define these environment variables (**bold** are mandatory): 
| *GREP_BIN* | `grep` | Path to *grep* binary in deployment target host. |

| *OMIT_CLEAN_DEPLOY* | `0` | Leave at deployment target host all deployment resources after doing a deploy. Useful when using bind mounts or *Compose* secrets (pointing to static content in deployment resources) or you want to check sent contents. |

| *OMIT_STATUS_CHECK* | `0` | Bypass status check process after deploying services. Useful when you need to be fast. |

| *OMIT_WAITING_TO_CONVERGE* | `0` | Bypass waiting for relaunched service to converge (when all service tasks are running and healthy, if supported). Useful when you need to be fast. Available only for *relaunch* action. |

| *REGISTRY_PASS* | - | Docker registry password, corresponding to a user with read permissions. **Required** for private registry or repository. |

| *REGISTRY_URL* | - | Docker registry address, where Docker must log in to retrieve images. Useful only when using private registry or repository. Default is empty, to use Docker Hub registry. |

| *REGISTRY_USER* | - | Docker registry username, corresponding to a user with read permissions. **Required** for private registry or repository. |
@@ -74,16 +168,19 @@ You may define these environment variables (**bold** are mandatory): 
| *STATUS_CHECK_INTERVAL* | `20` | Seconds to wait between check iterations. |

| *STATUS_CHECK_MIN_HITS* | `3` | Minimum number of successful checks to consider deployment as successful. |

| *STATUS_CHECK_RETRIES* | `10` | Maximum number of checks before considering deployment as failed. |

| *SWARM_RESOLVE_IMAGE* | `always` | Allows to edit the behaviour of the registry query to resolve image digests and supported platforms (`always`, `changed` or `never`). |

| *USE_IMAGE_DIGEST* | `0` | Update service image using digest data when relaunching. Available only for *relaunch* action. |

| *SWARM_RESOLVE_IMAGE* | `always` | Allows to edit the behaviour of *deploy* and *relaunch* actions when querying registry, to resolve image digests and supported platforms or not. Supported values are `always`, `changed` or `never`. |

| *USE_IMAGE_DIGEST* | `0` | Update service image using digest data when relaunching. Useful when using images which receive updates under same tag and want to keep them updated with same version on all nodes. Available only for *relaunch* action. |



### Your services



When using *deploy* action, you can configure your own services through variables:

When using *deploy* action, you can configure your own services through variables.



> Note that you must declare them at your compose files too (into `environment` section, for example).



* Define any variable whose name is prefixed by `ENV_PREFIX` prefix:

  1. Set variable `docker run ... -e DD_ANY_NAME=value ... deploy`.

  2. `ANY_NAME` will be available into service containers with `value` value.



* Pass any variable as deploy script parameter (without `ENV_PREFIX` prefix):

  1. Set parameter to deploy script: `docker run ... deploy ANY_NAME=value`.

  2. `ANY_NAME` will be available into service containers with `value` value.
@@ -131,4 +228,4 @@ $ docker run --rm --name docker-deploy \ 
   * identified as `user`

   * authenticated through a RSA-1024 private key

   * into `example` stack

   * with `VARIABLE_1` and `VARIABLE_2` set in service
 
   * with `VARIABLE_1` and `VARIABLE_2` available to set in service

script/_check-config.sh

+41 −5
Original line number Diff line number Diff line
@@ -5,22 +5,58 @@ echo -e " ${INFO_COLOR}compose files [ ${DATA_COLOR}${COMPOSE_FILE}${INFO_COLOR 
echo -en "  ${INFO_COLOR}check command [ ${DATA_COLOR}"



# Antes de continuar, se comprueba que la configuración de despliegue sea válida para compose o swarm.

tempEnvFile=".env-config"

if [ ${docker23CompatibleTarget} -eq 0 ] && [ ${deployingToSwarm} -eq 0 ]

then

	echo -e "docker stack config${INFO_COLOR} ]${NULL_COLOR}\n"

	grep -v '^[#| ]' "${COMPOSE_ENV_FILE_NAME}" | sed -r "s/(\w+)=(.*)/export \1='\2'/g" > .env-config



	grep -v '^[#| ]' "${COMPOSE_ENV_FILE_NAME}" | sed -r "s/(\w+)=(.*)/export \1='\2'/g" > "${tempEnvFile}"



	env -i /bin/sh -c "\

		. $(pwd)/.env-config && \

		rm $(pwd)/.env-config && \

		. $(pwd)/${tempEnvFile} && \

		/usr/local/bin/docker stack config -c ${swarmComposeFileSplitted} > /dev/null"

else

	echo -e "docker compose config${INFO_COLOR} ]${NULL_COLOR}\n"



	docker compose config -q

	if [ ${ALLOW_COMPOSE_ENV_FILE_INTERPOLATION} -eq 0 ]

	then

		while IFS= read -r envLine

		do

			if [ -z "${envLine}" ] || echo "${envLine}" | grep -q '^[#| ]'

			then

				continue

			else

				variableName=$(echo "${envLine}" | cut -d '=' -f 1)

				variableValue=$(echo "${envLine}" | cut -d '=' -f 2-)



				# Si la variable ya tiene valor entrecomillado simple o los dólares duplicados, se usa tal cual

				if echo "${variableValue}" | grep -q '\$\$' || echo "${variableValue}" | grep -q "^'"

				then

					envConfigContent="${envConfigContent}${variableName}=${variableValue}\\n"

				else

					envConfigContent="${envConfigContent}${variableName}='${variableValue}'\\n"

				fi

			fi

		done < "${COMPOSE_ENV_FILE_NAME}"

		echo -e "${envConfigContent}" > "${tempEnvFile}"



		# Si se va a desplegar con modo compose, se preservan las variables con interpolación omitida

		if [ ${deployingToSwarm} -ne 0 ]

		then

			cp -a "${tempEnvFile}" "${COMPOSE_ENV_FILE_NAME}"

		fi

	else

		cp -a "${COMPOSE_ENV_FILE_NAME}" "${tempEnvFile}"

	fi



	docker compose --env-file "${tempEnvFile}" config -q

fi



checkConfigExitCode=${?}



rm "${tempEnvFile}"



if [ ${?} -eq 0 ]

if [ ${checkConfigExitCode} -eq 0 ]

then

	echo -e "${PASS_COLOR}Valid compose configuration!${NULL_COLOR}"

else

script/_check-deploy.sh

+1 −1
Original line number Diff line number Diff line
@@ -104,7 +104,7 @@ checkDeployCmd="\ 
		fi ; \

	done"



if ssh ${SSH_PARAMS} "${SSH_REMOTE}" "${checkDeployCmd}"

if runRemoteCmd "${checkDeployCmd}"

then

	echo -e "\n${PASS_COLOR}All services seems ok!${NULL_COLOR}"

else

script/_check-env.sh

+4 −41
Original line number Diff line number Diff line
@@ -2,7 +2,7 @@ 


# Se comprueba si está disponible el binario docker en el entorno donde se va a desplegar.

checkDockerCmd="command -v docker > /dev/null"

if ! ssh ${SSH_PARAMS} "${SSH_REMOTE}" ${checkDockerCmd}

if ! runRemoteCmd "${checkDockerCmd}"

then

	echo -e "\n${FAIL_COLOR}Docker is not available at deployment target host environment!${NULL_COLOR}"

	eval "${closeSshCmd}"
@@ -10,8 +10,8 @@ then 
fi



# Se obtiene la versión de Docker disponible en el entorno donde se va a desplegar.

getDockerVersionCmd="docker version --format '{{.Client.Version}}'"

dockerVersion=$(ssh ${SSH_PARAMS} "${SSH_REMOTE}" ${getDockerVersionCmd})

getDockerVersionCmd="docker version --format '{{.Server.Version}}'"

dockerVersion=$(runRemoteCmd "${getDockerVersionCmd}")



echo -e "  ${INFO_COLOR}host Docker version [ ${DATA_COLOR}${dockerVersion}${INFO_COLOR} ]${NULL_COLOR}"
@@ -26,43 +26,6 @@ then 
	deployingToSwarm=1

else

	checkSwarmManagerAvailabilityCmd="[ \$(docker info --format '{{.Swarm.ControlAvailable}}') = true ]"

	ssh ${SSH_PARAMS} "${SSH_REMOTE}" ${checkSwarmManagerAvailabilityCmd}

	runRemoteCmd "${checkSwarmManagerAvailabilityCmd}"

	deployingToSwarm=${?}

fi
 


if [ ${deployingToSwarm} -eq 0 ]

then

	deploymentTypeLabel="Swarm"



	# Prepara los argumentos necesarios para indicar los ficheros compose a usar, para swarm o para compose.

	swarmComposeFileSplitted=$(echo ${COMPOSE_FILE} | sed 's/:/ -c /g')

else

	deploymentTypeLabel="Compose"



	# Prepara los argumentos necesarios para indicar los ficheros compose a usar, para compose.

	standardComposeFileSplitted=$(echo ${COMPOSE_FILE} | sed 's/:/ -f /g')



	# Se comprueba si está disponible el plugin compose de docker o el antiguo binario docker-compose.

	if [ ${docker23CompatibleTarget} -eq 0 ]

	then

		getComposeVersionPrefixCmd="docker compose"

	else

		getComposeVersionPrefixCmd="docker-compose"



		checkDockerComposeBinaryCmd="command -v docker-compose > /dev/null"

		if ! ssh ${SSH_PARAMS} "${SSH_REMOTE}" ${checkDockerComposeBinaryCmd}

		then

			echo -e "\n${FAIL_COLOR}Legacy docker-compose binary is not available at deployment target host environment!${NULL_COLOR}"

			eval "${closeSshCmd}"

			exit 1

		fi

	fi



	# Se obtiene la versión de Docker Compose disponible en el entorno donde se va a desplegar.

	getComposeVersionCmd="${getComposeVersionPrefixCmd} version --short"

	composeVersion=$(ssh ${SSH_PARAMS} "${SSH_REMOTE}" ${getComposeVersionCmd})



	echo -e "  ${INFO_COLOR}host Docker Compose version [ ${DATA_COLOR}${composeVersion}${INFO_COLOR} ]${NULL_COLOR}"

fi



echo -e "  ${INFO_COLOR}deployment type [ ${DATA_COLOR}${deploymentTypeLabel}${INFO_COLOR} ]${NULL_COLOR}"