Instalación de MooseFS

Sistema distribuido: MooseFS

MODELO 1:

HA con 3 Masters + 4 Chunks + 2 Metadata

Arquitectura:

3 Master servers
4 Chunk servers
2 Metaloggers
VIP gestionada por Keepalived con VRRP
HAProxy como frontend TCP

Direccionamiento

VIP: 10.200.3.50
Masters: 10.200.3.51 / 10.200.3.52 / 10.200.3.53
Chunks: 10.200.3.55 / 56 / 57 / 58
Metaloggers: 10.200.3.71 / 72

Instalación (común para todo el stack)

mkdir -p /etc/apt/keyrings
curl https://repository.moosefs.com/moosefs.key | gpg -o /etc/apt/keyrings/moosefs.gpg --dearmor

Ejecutamos apt update

apt update

E instalamos

echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/moosefs.gpg] http://repository.moosefs.com/moosefs-4/apt/debian/trixie trixie main" > /etc/apt/sources.list.d/moosefs.list

MASTER ACTIVO (10.200.3.51)

/etc/mfs/mfsmaster.cfg

PERSONALITY = master
WORKING_USER = mfs
WORKING_GROUP = mfs
DATA_PATH = /var/lib/mfs
BACK_META_KEEP_PREVIOUS = 3
REPLICATIONS_DELAY_INIT = 300

/etc/mfs/mfsexports.cfg

10.200.3.0/24 / rw,alldirs,admin,maproot=0

Inicializar:

mfsmaster -a
systemctl enable moosefs-master
systemctl start moosefs-master

SHADOW MASTERS (.52 y .53)

PERSONALITY = shadow
MASTER_HOST = 10.200.3.50
MASTER_PORT = 9419
DATA_PATH = /var/lib/mfs

CHUNK SERVERS

apt install moosefs-chunkserver -y

Inicializamos las carpetas o discos de los chunks

mkdir -p /data/mfschunk
chown mfs:mfs /data/mfschunk

Editamos el fstab para añadir el resto de discos

mkdir -p /mnt/disk[X]/mfschunk
chown mfs:mfs /mnt/disk[X]/mfschunk

/etc/mfs/mfschunkserver.cfg

MASTER_HOST = 10.200.3.50
MASTER_PORT = 9420
DATA_PATH = /data/mfschunk, /mnt/disk[X]/mfschunk
WORKING_USER = mfs
WORKING_GROUP = mfs

/etc/mfs/mfshdd.cfg

/data/mfschunk
/mnt/disk[X]/mfschunk

En este apartado añadiremos los diferentes discos que tengan los servidores de chunk

systemctl enable moosefs-chunkserver
systemctl start moosefs-chunkserver

Repetir en .55, .56, .57, .58.

Metadata backup servers (10.200.3.71 y .72)

apt install moosefs-master -y

/etc/mfs/mfsmaster.cfg

PERSONALITY = metalogger
MASTER_HOST = 10.200.3.50
MASTER_PORT = 9419
DATA_PATH = /var/lib/mfs

systemctl enable moosefs-master
systemctl start moosefs-master

Keepalived (nodo principal)

vrrp_instance MFS_VIP {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 200
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass mfscluster
    }

    virtual_ipaddress {
        10.200.3.50/24
    }
}

Nodo secundario (52)

vrrp_script chk_haproxy {
    script "/usr/local/bin/check_haproxy.sh"
    interval 2
    weight 2
}

vrrp_instance MFS_VIP {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass mfscluster
    }

    virtual_ipaddress {
        10.200.3.50/24
    }

    track_script {
        chk_haproxy
    }
}

Tercer nodo (53)

vrrp_script chk_haproxy {
    script "/usr/local/bin/check_haproxy.sh"
    interval 2
    weight 2
}

vrrp_instance MFS_VIP {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass mfscluster
    }

    virtual_ipaddress {
        10.200.3.50/24
    }

    track_script {
        chk_haproxy
    }
}

HAProxy

frontend mfs_master_front
    bind 0.0.0.0:9419
    default_backend mfs_master_back

backend mfs_master_back
    option tcp-check
    tcp-check connect port 9419
    server master1 10.200.3.51:9419 check
    server master2 10.200.3.52:9419 check
    server master3 10.200.3.53:9419 check

En la versión pro los nodos master, se promocionan solos, en la versión community, hay que hacer un script.

#!/bin/bash
MASTER_IP="10.200.3.50"
MASTER_PORT="9419"

timeout 2 bash -c "</dev/tcp/${MASTER_IP}/${MASTER_PORT}" &>/dev/null
if [ $? -ne 0 ]; then
    systemctl stop moosefs-master
    sed -i 's/PERSONALITY *= *shadow/PERSONALITY = master/' /etc/mfs/mfsmaster.cfg
    mfsmaster -o
    systemctl start moosefs-master
fi

Y lo dejamos como ejecutable

chmod +x /usr/local/bin/mfs-auto-promote.sh

MODELO 2 - 1 Master + 4 Chunks

Master: 10.200.3.51
Chunks: 10.200.3.55 / 56 / 57 / 58

MASTER

PERSONALITY = master
WORKING_USER = mfs
WORKING_GROUP = mfs
DATA_PATH = /var/lib/mfs

10.200.3.0/24 / rw,alldirs,admin,maproot=0

CHUNKS

MASTER_HOST = 10.200.3.51
MASTER_PORT = 9420
DATA_PATH = /data/mfschunk
WORKING_USER = mfs
WORKING_GROUP = mfs

Cliente

mfsmount /mnt/mfs -H 10.200.3.51

Replicación recomendada

mfssetgoal 2 /