rpi_docker_home_server/README.org

* OpenGardenCloud ** Flash Hypriot You can check last images http://blog.hypriot.com/downloads/][here and use https://github.com/hypriot/flash][flash tool to flash your RaspberryPi SD: #+begin_example flash --hostname your-hostname https://github.com/hypriot/image-builder-rpi/releases/download/v1.4.0/hypriotos-rpi-v1.4.0.img.zip #+end_example SSH into each RPI: #+begin_example ssh pirate@you-rpi-ip #+end_example As of version 1.4, default credentials are pirate/hypriot. You can use arp-scan to guess the IP. You can also use: #+begin_example function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) } #+end_example Change default password: #+begin_example passwd #+end_example You can also set up paswordless access with: #+begin_example ssh-copy-id -i ~/.ssh/your-key_rsa.pub pirate@your-rpi -o "IdentitiesOnly yes" #+end_example And also add an entry to you ~/.ssh/config file: #+begin_example Host your-rpi-1 your-rpi-2 ... Hostname %h.local User pirate IdentityFile ~/.ssh/your-key_rsa IdentitiesOnly yes StrictHostKeyChecking no #+end_example If you want, you can also add this config snippet to all your nodes and add your private key to each =~/.ssh= folder to be able to connect from one RPI to another. (?) Add regular user to docker group #+begin_example sudo usermod -aG docker pirate #+end_example (Optional) In case you see annoying warning messages about locales from perl: #+begin_example sudo dpkg-reconfigure locales #+end_example (Optional) Install some useful packages #+begin_example sudo aptitude update && sudo aptitude install rsync zsh #+end_example ** (Optional) Encrypt external hard disk #+begin_example sudo aptitude install cryptsetup #sudo fdisk /dev/sdX sudo parted /dev/sdX sudo cryptsetup --verify-passphrase luksFormat /dev/sdX1 -c aes -s 256 -h sha256 sudo cryptsetup luksOpen /dev/sdX1 volumes sudo mkfs -t ext4 -m 1 -O dir_index,sparse_super /dev/mapper/volumes sudo mkdir -p /media/volumes #sudo mount -t auto /dev/mapper/volumes /media/volumes sudo dd if=/dev/urandom of=/root/volumes_luks_pwd bs=1024 count=4 sudo chmod 0400 /root/volumes_luks_pwd sudo cryptsetup luksAddKey /dev/sdX1 /root/volumes_luks_pwd #+end_example Add to =/etc/crypttab=: #+begin_example volumes /dev/disk/by-uuid/uuid-of-your-drive /root/volumes_luks_pwd luks #+end_example and add to =/etc/fstab=: #+begin_example /dev/mapper/volumes /media/volumes ext4 defaults 0 2 #+end_example ** NFS Install server on main host: #+begin_example sudo aptitude install nfs-kernel-server sudo mkdir -p /export/volumes sudo mount --bind /media/volumes /export/volumes #+end_example And add the following line to =/etc/fstab= to avoid repeating it on startup: #+begin_example /media/volumes /export/volumes none bind 0 0 #+end_example And to =/etc/exports=: #+begin_example /export 192.168.1.0/24(rw,fsid=0,insecure,no_subtree_check,async) /export/volumes 192.168.1.0/24(rw,nohide,insecure,no_subtree_check,async,no_root_squash) #+end_example (changing network/mask by your local values) On the other nodes: #+begin_example sudo aptitude install nfs-common #+end_example And add to =/etc/fstab=: #+begin_example your-main-host:/export/volumes /media/volumes nfs auto,user 0 0 #+end_example ** Swap file http://jermsmit.com/my-raspberry-pi-needs-a-swap/ #+begin_example dd if=/dev/zero of=/media/volumes/swap bs=1M count=2048 chmod 600 /media/volumes/swap mkswap /media/volumes/swap swapon /media/volumes/swap #+end_example Add to /etc/fstab: #+begin_example /media/volumes/swap swap swap defaults 0 0 #+end_example Repeat for worker nodes (changing name of swap file) ** Avahi When the dockers are running, some service users (e.g. =dovecot= or =mysqld=) can have conflicting ids with the one of avahi, making it fail. To avoid that, we can just increase its =uid=, e.g.: #+begin_example sudo systemctl stop avahi-daemon sudo usermod -u 205 avahi sudo systemctl restart dbus sudo systemctl start avahi-daemon #+end_example Install missing =libnss-mdns= package (see explanation https://paulnebel.io/api/containers/lean/node/raspberry_pi/swarm/2016/08/23/hypriotos-swarm-raspberry-pi-cluster/][here): #+begin_example sudo aptitude install libnss-mdns #+end_example Also make sure =avahi-daemon= works, and otherwise restart it. See https://github.com/hypriot/image-builder-rpi/issues/170][this issue. ** Data and volumes If you have existing data, create folders (otherwise setup script will do it) and copy it data: #+begin_example sudo mkdir -p /media/volumes/mail/ sudo mkdir -p /media/volumes/nextcloud sudo chown -R pirate:pirate /media/volumes/* sudo mkdir -p /media/volumes/openldap/data sudo mkdir -p /media/volumes/openldap/config sudo mkdir -p /media/volumes/openldap/certs sudo chown -R 999 /media/volumes/openldap* #+end_example From your current installation: #+begin_example rsync -auv --delete -e "ssh -i ~/.ssh/your-key_rsa" /var/www/nextcloud/data your-main-host:/media/volumes/nextcloud/ mysqldump --lock-tables -u nextcloud -p -h localhost nextcloud > /var/www/nextcloud/nextcloud_db_backup.sql scp -i ~/.ssh/your-key_rsa /var/www/nextcloud/nextcloud_db_backup.sql your-main-host:/media/volumes/nextcloud/data/ rsync -auv --delete -e "ssh -i ~/.ssh/your-key_rsa" /srv/vmail/ your-main-host:/media/volumes/mail #+end_example ** Configuration and deployment If it's a restart, clean first previous containers: #+begin_example for i in $(docker ps -a | grep Exited | grep ogc | cut -f 1 -d ' '); do docker rm $i; done; #+end_example If it's a new installation, run setup scripts: #+begin_example ./set_arch.sh amd64 ./setup.sh # -y for default values #+end_example Optionally build: #+begin_example docker-compose build #+end_example And then restart: #+begin_example docker-compose --compatibility -p ogc up -d #+end_example Note: =ogc= is just a custom prefix to easily identify containers, you can use your own. Add users: #+begin_example ./add_users.sh #+end_example Add DNS entries: #+begin_example ./add_dns_entries.sh #+end_example Add Nextcloud apps: #+begin_example ./nextcloud_apps_after_update.sh #+end_example If you add or modify a service, you can update it running: #+begin_example docker-compose build && docker-compose -p ogc up -d #+end_example If you want to re-create an image and restart the service you can run: #+begin_example docker-compose --compatibility -p ogc up -d --no-deps --build #+end_example ** Openldap #+begin_example ldapsearch -x -w your-admin-ldap-password -D cn=admin,dc=your-domain,dc=com -b dc=your-domain,dc=com -LLL #+end_example To reset a user's password: Copy this into a file, =user_pwd.ldif=: #+begin_example dn: uniqueIdentifier=your-user,ou=people,dc=your-domain,dc=com changetype: modify replace: userPassword userPassword: {SSHA}Djpd2d+kbQm4ftHupSaS65wl8l8EbDot #+end_example And the run: #+begin_example ldapadd -W -D "cn=admin,dc=your-domain,dc=com" -f user_pwd.ldif #+end_example You can generate the password with: #+begin_example slappasswd -s your-password #+end_example You can use the following script to add users if you have previously created =ldif= files: #+begin_example ./add_users.sh #+end_example ** MariaDB If you have existing data, make sure root password matches and access from outside ('%') is allowed. ** Nextcloud After first run, set DATA_CHOWN=0. Otherwise every time you deploy the whole folder with all your data will be recursed to change ownership, and it can take long when it's only needed for the first time. ** Let's Encrypt If you want to add more domains after deployment, you can run this command manually from HAProxy docker instance (see https://serversforhackers.com/c/letsencrypt-with-haproxy][this): #+begin_example sudo certbot certonly --standalone -d nextcloud.example.com -d git.example.com \ --non-interactive --agree-tos --email admin@example.com \ --http-01-port=8888 #+end_example Notice that when updating your certificate, you will need to restart haproxy container, due to https://stackoverflow.com/a/50480260/1937418][this issue. You can do it with: #+begin_example docker exec -ti ogc_haproxy_1 /etc/cron.daily/letsencrypt && docker stop ogc_haproxy_1 && docker-compose -p ogc up -d haproxy #+end_example ** Own registry Follow the instructions https://docs.docker.com/engine/swarm/stack-deploy/#set-up-a-docker-registry][here to set up your own registry: #+begin_example docker service create --name registry --publish published=5000,target=5000 registry:2 #+end_example ** Dynamic DNS Check your domain registration provider ** Fail2ban Install fail2ban in you docker swarm master node if you want to allow ssh connections from outside. #+begin_example sudo aptitude install fail2ban #+end_example Have a look at the http://www.fail2ban.org/wiki/index.php/MANUAL_0_8][documentation for configuration. ** Port mapping Get into your router admin page and redirect ports: - =80=, =443= for Web (Nextcloud and eventually other through HaProxy) - =25=, =143=, =587=, =993= for mail server - =22= for ssh to your docker swarm master node IP. ** TODO - Install and enable Nextcloud apps automatically - XMPP - VPN - Open social networks - Transmission - Alternative: run your own registry for images.