Merge branch 'dev' into 'master' (98027600) · Commits · REDMIC Project / Storage / SeaweedFS

README.md

+152 −0

Original line number	Diff line number	Diff line
		# SeaweedFS

		Deployment of SeaweedFS, a distributed object store and file system

		## Tareas comunes

		### Cambio de replicación y rebalanceo de volúmenes

		Ejecutar desde un nodo `volume`, por ejemplo:

		```sh
		weed shell -master swfs-master-1:9333,swfs-master-2:9333,swfs-master-3:9333

		lock

		# cambia ajuste de replicación por defecto
		volume.configure.replication -replication=100

		# aplica cambio de replicación
		volume.fix.replication

		# reparte equitativamente los volúmenes entre los nodos
		volume.balance -force

		unlock

		exit
		```

		### Migración de store usado por filer

		Ejecutar desde un nodo `filer`.

		Primero, se guarda un backup del store actual:

		```sh
		weed shell -master swfs-master-1:9333,swfs-master-2:9333,swfs-master-3:9333 -filer 127.0.0.1:8888 -filerGroup filesystem

		# fija directorio dentro de swfs desde el que guardar
		fs.cd /

		# genera fichero .meta con el backup del store
		fs.meta.save

		exit
		```

		Tras relanzar el servicio con el nuevo store, se carga el backup:

		```sh
		weed shell -master swfs-master-1:9333,swfs-master-2:9333,swfs-master-3:9333 -filer 127.0.0.1:8888 -filerGroup filesystem

		# carga fichero de backup en el nuevo store
		fs.meta.load 127.0.0.1:8888-*.meta

		exit

		# elimina fichero backup
		rm 127.0.0.1:8888-*.meta
		```

		### Mantenimiento de store etcd (en desuso)

		Ejecutar desde un nodo `etcd`:

		```sh
		# listar miembros del cluster con sus campos
		etcdctl member list -w fields

		# listar alarmas lanzadas
		etcdctl alarm list

		# comprobar estado del servicio local en detalle
		etcdctl --write-out=table endpoint status

		# capturar el número de la última revisión
		rev=$(etcdctl --endpoints=:2379 endpoint status --write-out="json" \| egrep -o '"revision":[0-9]' \| egrep -o '[0-9].')

		# compactar las revisiones anteriores a la capturada
		etcdctl compact $rev

		# desfragmentar el espacio tras la compactación
		etcdctl defrag

		# eliminar las alarmas para salir del modo mantenimiento
		etcdctl alarm disarm
		```

		## Plugin de volúmenes Docker

		### Compilación e instalación del plugin

		Se ha hecho uso del plugin [katharostech/docker-plugin_seaweedfs](https://github.com/katharostech/docker-plugin_seaweedfs/tree/fb624481819998223bb2043171caaea1e29a4a7f), concretamente en el commit `fb62448` donde se utiliza seaweedfs en su versión `3.43`. Esto nos aporta mejoras en cuanto a compatibilidad y rendimiento, pero con la pega de que no se ha actualizado su versión compilada ([katharostech/seaweedfs-volume-plugin](https://registry.hub.docker.com/r/katharostech/seaweedfs-volume-plugin)) para su uso directo. Ya está disponible en [redmic/seaweedfs-volume-plugin](https://registry.hub.docker.com/r/redmic/seaweedfs-volume-plugin).

		Por ello, se hace necesaria la descarga del repositorio (también está disponible una copia estática del commit `fb62448` en `plugins/seaweedfs-volume-plugin`), la compilación del plugin y la subida del resultado para su uso en las diferentes máquinas:

		```sh
		git clone https://github.com/katharostech/docker-plugin_seaweedfs.git
		cd docker-plugin_seaweedfs
		make clean rootfs

		# si no se ha hecho antes, identificarse en DockerHub
		docker login

		docker plugin create redmic/seaweedfs-volume-plugin:v3.43 ./plugin
		docker plugin push redmic/seaweedfs-volume-plugin:v3.43
		docker plugin rm redmic/seaweedfs-volume-plugin:v3.43

		docker plugin create redmic/seaweedfs-volume-plugin:latest ./plugin
		docker plugin push redmic/seaweedfs-volume-plugin:latest
		docker plugin rm redmic/seaweedfs-volume-plugin:latest

		make clean
		```

		Luego, en cada nodo se puede instalar mediante:

		```sh
		docker plugin install --alias seaweedfs redmic/seaweedfs-volume-plugin:latest \
		HOST=localhost:8888 \
		REMOTE_PATH=/docker-volumes \
		MOUNT_OPTIONS="-volumeServerAccess=publicUrl -dataCenter=dc-$(docker node inspect -f '{{.ID}}' self) -cacheDir=/tmp -cacheCapacityMB=2048"
		```

		O si ya estaba instalado, pero se desea actualizar o cambiar a esta compilación:

		```sh
		docker plugin disable -f seaweedfs:latest

		docker plugin upgrade --grant-all-permissions seaweedfs:latest redmic/seaweedfs-volume-plugin:latest

		docker plugin enable seaweedfs:latest
		```

		### Configuración del plugin

		Para ajustar la configuración del plugin cuando ya estaba funcionando:

		```sh
		docker plugin disable -f seaweedfs:latest

		docker plugin set seaweedfs:latest \
		HOST=127.0.0.1:8888 \
		MOUNT_OPTIONS="..."

		docker plugin enable seaweedfs:latest
		```

		Las variables disponibles son:

		* `HOST`: Referencia al servicio `filer`, en este caso siempre ha de ser local.
		* `MOUNT_OPTIONS`: Acepta cualquier parámetro de `weed mount`, ver su salida de ayuda para más información.
		* `REMOTE_PATH`: Ruta raíz donde almacenar los volúmenes dentro de seaweedfs.
		* `ROOT_VOLUME_NAME`: Nombre de volumen global con todo el contenido. Si se deja vacío no se genera.
		* `LOG_LEVEL`: Nivel de salida de logs generados por el plugin. Consultar desde la máquina con `journalctl -f -n 100 -u docker`.

deploy/.env

+8 −14

Original line number	Diff line number	Diff line
		@@ -4,15 +4,20 @@ IMAGE_TAG=latest
		MASTER_PORT=9333
		MASTER_IP=swfs-master-{{.Task.Slot}}
		MASTER_VOLUME_SIZE_LIMIT_MB=1024
		DEFAULT_REPLICATION=020
		MASTER_DEFAULT_REPLICATION=200
		MASTER_GARBAGE_THRESHOLD=0.01

		VOLUME_PORT=8080
		VOLUME_IP=swfs-volume-{{.Node.ID}}
		VOLUME_MAX=0
		VOLUME_DISK=hdd
		VOLUME_FILE_SIZE_LIMIT_MB=256
		VOLUME_UPLOAD_LIMIT_MB=256
		VOLUME_DOWNLOAD_LIMIT_MB=256
		VOLUME_PUBLIC_HOST={{.Node.Hostname}}
		VOLUME_READ_MODE=redirect
		VOLUME_READ_BUFFER_SIZE=4
		VOLUME_HAS_SLOW_READ=true
		PUBLIC_VOLUME_PORT=8080
		VOLUME_PORT_MODE=host

		@@ -29,23 +34,16 @@ FILER_PORT_MODE=host

		MASTER_HOST=swfs-master
		DATA_PATH=/data
		DATACENTER=dc
		DATACENTER=dc-{{.Node.ID}}
		RACK=r-{{.Node.ID}}
		METRICS_PORT=9090

		ETCD_DATA_PATH=/etcd-data
		ETCD_CLIENT_PORT=2379
		ETCD_PEER_PORT=2380
		ETCD_IP=swfs-etcd-{{.Task.Slot}}
		ETCD_HOST=swfs-etcd
		ETCD_CLUSTER_STATE=new
		ETCD_CLUSTER_TOKEN=changeme

		HEALTHCHECK_INTERVAL=15s
		HEALTHCHECK_TIMEOUT=60s
		HEALTHCHECK_RETRIES=10
		HEALTHCHECK_START_PERIOD=2m

		TRAEFIK_ENABLE=true
		TRAEFIK_ENTRYPOINT=http
		PUBLIC_HOSTNAME=change.me
		MASTER_TRAEFIK_SUBDOMAIN=swfs-master
		@@ -58,7 +56,3 @@ FILER_TRAEFIK_AUTH=user:$apr1$md5password
		MASTER_VOL_NAME=seaweedfs-master-{{.Task.Slot}}-vol
		VOLUME_VOL_NAME=seaweedfs-volume-{{.Node.ID}}-vol
		FILER_VOL_NAME=seaweedfs-filer-{{.Node.ID}}-vol
		ETCD_VOL_NAME=seaweedfs-etcd-{{.Node.ID}}-vol

		MASTER_NODE_LABEL_NAME=swfs-master
		MASTER_NODE_LABEL_VALUE=1

deploy/docker-compose.tmpl.yml

+42 −70

Original line number	Diff line number	Diff line
		@@ -11,8 +11,9 @@ services:
		weed -logtostderr=true master
		-mdir=${DATA_PATH}
		-volumePreallocate
		-volumeSizeLimitMB=${MASTER_VOLUME_SIZE_LIMIT_MB}
		-defaultReplication=${DEFAULT_REPLICATION}
		-volumeSizeLimitMB=$${MASTER_VOLUME_SIZE_LIMIT_MB}
		-defaultReplication=$${MASTER_DEFAULT_REPLICATION}
		-garbageThreshold=$${MASTER_GARBAGE_THRESHOLD}
		-ip=$${MASTER_IP}
		-ip.bind=0.0.0.0
		-port=${MASTER_PORT}
		@@ -22,6 +23,9 @@ services:
		hostname: ${MASTER_IP}
		environment:
		MASTER_IP:
		MASTER_VOLUME_SIZE_LIMIT_MB:
		MASTER_DEFAULT_REPLICATION:
		MASTER_GARBAGE_THRESHOLD:
		networks:
		seaweedfs-net:
		traefik-net:
		@@ -42,7 +46,7 @@ services:
		delay: ${MASTER_UPDATE_DELAY:-1m}
		placement:
		constraints:
		- "node.labels.${MASTER_NODE_LABEL_NAME}==${MASTER_NODE_LABEL_VALUE}"
		- "node.labels.${MASTER_NODE_LABEL_NAME:-swfs-master}==${MASTER_NODE_LABEL_VALUE:-1}"
		max_replicas_per_node: 1
		resources:
		limits:
		@@ -52,7 +56,7 @@ services:
		cpus: '${MASTER_RESOURCES_RESERVATIONS_CPUS:-0.01}'
		memory: ${MASTER_RESOURCES_RESERVATIONS_MEMORY:-64M}
		labels:
		traefik.enable: 'true'
		traefik.enable: ${TRAEFIK_ENABLE}
		traefik.http.routers.swfs-master.entrypoints: ${TRAEFIK_ENTRYPOINT}
		traefik.http.routers.swfs-master.rule: Host(`${MASTER_TRAEFIK_SUBDOMAIN}.${PUBLIC_HOSTNAME}`)
		traefik.http.middlewares.swfs-master-auth.basicauth.users: ${MASTER_TRAEFIK_AUTH}
		@@ -69,17 +73,20 @@ services:
		$$(echo "
		weed -logtostderr=true volume
		-dir=${DATA_PATH}
		-max=${VOLUME_MAX}
		-max=$${VOLUME_MAX}
		-dataCenter=$${VOLUME_DATACENTER}
		-rack=$${VOLUME_RACK}
		-disk=${VOLUME_DISK}
		-fileSizeLimitMB=${VOLUME_FILE_SIZE_LIMIT_MB}
		-concurrentUploadLimitMB=${VOLUME_UPLOAD_LIMIT_MB}
		-concurrentDownloadLimitMB=${VOLUME_DOWNLOAD_LIMIT_MB}
		-disk=$${VOLUME_DISK}
		-fileSizeLimitMB=$${VOLUME_FILE_SIZE_LIMIT_MB}
		-concurrentUploadLimitMB=$${VOLUME_UPLOAD_LIMIT_MB}
		-concurrentDownloadLimitMB=$${VOLUME_DOWNLOAD_LIMIT_MB}
		-ip=$${VOLUME_IP}
		-ip.bind=0.0.0.0
		-port=${VOLUME_PORT}
		-publicUrl=$${VOLUME_PUBLIC_HOST}:${PUBLIC_VOLUME_PORT}
		-readMode=$${VOLUME_READ_MODE}
		-readBufferSizeMB=$${VOLUME_READ_BUFFER_SIZE}
		-hasSlowRead=$${VOLUME_HAS_SLOW_READ}
		-mserver=${MASTER_HOST}-1:${MASTER_PORT},${MASTER_HOST}-2:${MASTER_PORT},${MASTER_HOST}-3:${MASTER_PORT}
		-metricsPort=${METRICS_PORT}
		")
		@@ -87,8 +94,16 @@ services:
		environment:
		VOLUME_IP:
		VOLUME_PUBLIC_HOST:
		VOLUME_MAX:
		VOLUME_DATACENTER: ${DATACENTER}
		VOLUME_RACK: ${RACK}
		VOLUME_DISK:
		VOLUME_FILE_SIZE_LIMIT_MB:
		VOLUME_UPLOAD_LIMIT_MB:
		VOLUME_DOWNLOAD_LIMIT_MB:
		VOLUME_READ_MODE:
		VOLUME_READ_BUFFER_SIZE:
		VOLUME_HAS_SLOW_READ:
		networks:
		seaweedfs-net:
		traefik-net:
		@@ -123,7 +138,7 @@ services:
		cpus: '${VOLUME_RESOURCES_RESERVATIONS_CPUS:-0.1}'
		memory: ${VOLUME_RESOURCES_RESERVATIONS_MEMORY:-320M}
		labels:
		traefik.enable: 'true'
		traefik.enable: ${TRAEFIK_ENABLE}
		traefik.http.routers.swfs-volume.entrypoints: ${TRAEFIK_ENTRYPOINT}
		traefik.http.routers.swfs-volume.rule: Host(`${VOLUME_TRAEFIK_SUBDOMAIN}.${PUBLIC_HOSTNAME}`)
		traefik.http.middlewares.swfs-volume-auth.basicauth.users: ${VOLUME_TRAEFIK_AUTH}
		@@ -141,11 +156,11 @@ services:
		weed -logtostderr=true filer
		-dataCenter=$${FILER_PREFERED_DATACENTER}
		-rack=$${FILER_PREFERED_RACK}
		-filerGroup=${FILER_GROUP}
		-collection=${FILER_COLLECTION}
		-maxMB=${FILER_MAX_MB}
		-concurrentUploadLimitMB=${FILER_UPLOAD_LIMIT_MB}
		-downloadMaxMBps=${FILER_DOWNLOAD_LIMIT_MB}
		-filerGroup=$${FILER_GROUP}
		-collection=$${FILER_COLLECTION}
		-maxMB=$${FILER_MAX_MB}
		-concurrentUploadLimitMB=$${FILER_UPLOAD_LIMIT_MB}
		-downloadMaxMBps=$${FILER_DOWNLOAD_LIMIT_MB}
		-ip=$${FILER_IP}
		-ip.bind=0.0.0.0
		-port=${FILER_PORT}
		@@ -158,9 +173,14 @@ services:
		FILER_IP:
		FILER_PREFERED_DATACENTER: ${DATACENTER}
		FILER_PREFERED_RACK: ${RACK}
		FILER_GROUP:
		FILER_COLLECTION:
		FILER_MAX_MB:
		FILER_UPLOAD_LIMIT_MB:
		FILER_DOWNLOAD_LIMIT_MB:
		WEED_LEVELDB2_ENABLED: 'false'
		WEED_ETCD_ENABLED: 'true'
		WEED_ETCD_SERVERS: http://${ETCD_HOST}-1:${ETCD_CLIENT_PORT},http://${ETCD_HOST}-2:${ETCD_CLIENT_PORT},http://${ETCD_HOST}-3:${ETCD_CLIENT_PORT}
		WEED_LEVELDB3_ENABLED: 'true'
		WEED_LEVELDB3_DIR: ${DATA_PATH}/filerldb3
		networks:
		seaweedfs-net:
		traefik-net:
		@@ -187,13 +207,13 @@ services:
		delay: ${FILER_UPDATE_DELAY:-15s}
		resources:
		limits:
		cpus: '${FILER_RESOURCES_LIMITS_CPUS:-1}'
		memory: ${FILER_RESOURCES_LIMITS_MEMORY:-128M}
		cpus: '${FILER_RESOURCES_LIMITS_CPUS:-2}'
		memory: ${FILER_RESOURCES_LIMITS_MEMORY:-256M}
		reservations:
		cpus: '${FILER_RESOURCES_RESERVATIONS_CPUS:-0.05}'
		memory: ${FILER_RESOURCES_RESERVATIONS_MEMORY:-64M}
		cpus: '${FILER_RESOURCES_RESERVATIONS_CPUS:-0.1}'
		memory: ${FILER_RESOURCES_RESERVATIONS_MEMORY:-128M}
		labels:
		traefik.enable: 'true'
		traefik.enable: ${TRAEFIK_ENABLE}
		traefik.http.routers.swfs-filer.entrypoints: ${TRAEFIK_ENTRYPOINT}
		traefik.http.routers.swfs-filer.rule: Host(`${FILER_TRAEFIK_SUBDOMAIN}.${PUBLIC_HOSTNAME}`)
		traefik.http.middlewares.swfs-filer-auth.basicauth.users: ${FILER_TRAEFIK_AUTH}
		@@ -201,50 +221,6 @@ services:
		traefik.http.services.swfs-filer.loadbalancer.server.port: ${FILER_PORT}
		traefik.http.services.swfs-filer.loadbalancer.sticky.cookie: 'true'

		swfs-etcd:
		image: ${ETCD_IMAGE_NAME:-quay.io/coreos/etcd}:${ETCD_IMAGE_TAG:-latest}
		entrypoint: /bin/sh
		command:
		- -c
		- \|
		$$(echo "
		/usr/local/bin/etcd
		--data-dir=${ETCD_DATA_PATH}
		--name $${ETCD_IP}
		--initial-advertise-peer-urls http://$${ETCD_IP}:${ETCD_PEER_PORT}
		--listen-peer-urls http://0.0.0.0:${ETCD_PEER_PORT}
		--advertise-client-urls http://$${ETCD_IP}:${ETCD_CLIENT_PORT}
		--listen-client-urls http://0.0.0.0:${ETCD_CLIENT_PORT}
		--initial-cluster ${ETCD_HOST}-1=http://${ETCD_HOST}-1:${ETCD_PEER_PORT},${ETCD_HOST}-2=http://${ETCD_HOST}-2:${ETCD_PEER_PORT},${ETCD_HOST}-3=http://${ETCD_HOST}-3:${ETCD_PEER_PORT}
		--initial-cluster-state ${ETCD_CLUSTER_STATE}
		--initial-cluster-token ${ETCD_CLUSTER_TOKEN}
		")
		hostname: ${ETCD_IP}
		environment:
		ETCD_IP:
		networks:
		seaweedfs-net:
		volumes:
		- etcd-vol:${ETCD_DATA_PATH}
		deploy:
		mode: replicated
		replicas: ${ETCD_REPLICAS:-3}
		restart_policy:
		delay: ${ETCD_RESTART_DELAY:-1s}
		update_config:
		delay: ${ETCD_UPDATE_DELAY:-10s}
		placement:
		constraints:
		- "node.labels.${MASTER_NODE_LABEL_NAME}==${MASTER_NODE_LABEL_VALUE}"
		max_replicas_per_node: 1
		resources:
		limits:
		cpus: '${ETCD_RESOURCES_LIMITS_CPUS:-1}'
		memory: ${ETCD_RESOURCES_LIMITS_MEMORY:-256M}
		reservations:
		cpus: '${ETCD_RESOURCES_RESERVATIONS_CPUS:-0.05}'
		memory: ${ETCD_RESOURCES_RESERVATIONS_MEMORY:-128M}

		networks:
		seaweedfs-net:
		name: ${SEAWEEDFS_NET_NAME:-seaweedfs-net}
		@@ -268,7 +244,3 @@ volumes:
		filer-vol:
		name: ${FILER_VOL_NAME}
		driver: local

		etcd-vol:
		name: ${ETCD_VOL_NAME}
		driver: local

plugins/seaweedfs-volume-plugin/.dockerignore

0 → 100644

+3 −0

Original line number	Diff line number	Diff line
		node_modules/
		plugin/
		.git/

plugins/seaweedfs-volume-plugin/.gitignore

0 → 100644

+2 −0

Original line number	Diff line number	Diff line
		node_modules/
		plugin/