Installing Portainer on a swarm

docker-swarm-portainer.md

This is how to install portainer on the swarm you created using this gist note this should work on non-swarm installs too!

Setup Portainer

Download the portain stack yaml that will do this for you

curl -L https://downloads.portainer.io/ce2-17/portainer-agent-stack.yml -o portainer-agent-stack.yml

now run it

sudo docker stack deploy -c portainer-agent-stack.yml portainer

this will create a single portainer container that runs on the management node and deploy the agent to all worker nodes, once complete you can now access portainer at http://docker-host-ip:9000 and define your admin users etc

You now have a fully function docker swarm with portainer and can deploy containers, stacks or templates.

How to use portainer is beyond the scope of this gist

This is the portainer install doc at time of writing for reference but the instructions above will work just fine

IF you mounting the filesystem at boot if so you need to make sure fstab doesn't try and mount them utill the network AND the ceph/glsuter services are full up and operational (not just started)

this will require you mess with service dependencies...

consider running a scripted mount command instead of relying on fstab

this was one reason I used the volume driver, can't recall if i modified docker service to start only once gluster was fully up

If thats not the issue maybe you have an inherent networking issue, do all your nodes have names registered in DNS and it might be worth putting all nodes in /etc/hosts - sometimes found gluster would rever to wanting to use nodename for no reason i could fathom

I used the following guide https://www.youtube.com/watch?v=Has6lUPdzzY
I've already added the nodes in /etc/hosts
How you do this ? IF you mounting the filesystem at boot if so you need to make sure fstab doesn't try and mount them utill the network AND the ceph/glsuter services are full up and operational (not just started)

Secondly, I have a 32GB disk on each of the nodes - 15GB goes to gluster - any idea how you can increase the amount for gluster ? And potential tweaks ?

I killed all the machines (which can happen after a long power surge). Then bringing them up and nothing works anymore. The scripted mount command, anything you documented ?

I can get it to work again when I run the mount -a command manually.
Strange thing is that last server booting up - it mounts correctly.

If only 1 server goes down, all work as expected. If 2 go down, the last one will mount correctly, but the first one doesn't.

Please note that I added the following to /etc/fstab
localhost:/staging-gfs /mnt glusterfs defaults,_netdev,backupvolfile-server=localhost 0 0

oh i don't use a scripted command, just suggesting things for you to try

all i can tell you is what I did works and with the exception of that one annoyance where i have to re-enable the plugin after router reboot (not node reboots), replacing gluster with a connection to my ceph is a project for the holidays. my docker VMs won't be the ceph nodes, the host those docker VMs sit on are the ceph nodes.

dont send me videos I am a grumpy early GenX' took me minutes to scrub through video, still couldn't figure out what the fuck he was doing, luckily he had good docs, took me seconds to read and undestand ;-)

yeah he fucked up and didn't include the automount, tl;dr don't rely on videos most of the time they don't really know what they are doing (or rather taking them as sign post, not a how to), alsogo to stack exchange, there are usually answers there from years ago

like this one
https://serverfault.com/questions/800494/glusterfs-mount-on-boot-on-clustered-servers-rhel-7

scroll down, you will see a script example

you will see the use of automount at the end of the glsuter fastab line.... he missed that.... this should mean the system won't try and mount until gluster is running

if that still fails you will need to start editing unit files, not fun - but folks seem to have posted those approaches too, always start with the most upvoted answer (in this case the automount directives), on linux its hard to know which is right approach as each distro works differently, i only use debian for my docker hosts for a good reason

if it helps this is what i have in my fstab on all 3 nodes

#for glusterfs
UUID=8a5eeebb-a268-45ac-a2c7-e2cef9b2d8d2 /mnt/glusterfs xfs  defaults  0  0

#for gluster vol1
localhost:/gluster-vol1  /mnt/gluster-vol1 glusterfs  defaults,_netdev,noauto,x-systemd.automount        0 0

Hi mate,

I followed the following guide https://www.howtoforge.com/how-to-install-and-configure-glusterfs-on-ubuntu-22-04/#google_vignette together with your last post - and we got lift off !

Another question : how do you get that you can use

volumes:
  data:
    driver: gluster-vol1

Hi mate,

I followed the following guide https://www.howtoforge.com/how-to-install-and-configure-glusterfs-on-ubuntu-22-04/#google_vignette together with your last post - and we got lift off !

fabulous, but i am starting to think you didn't follow my documentation... the automount has always been in my docs, and you question about glutser-vol1 means i thnk you didn't read my gluster documentation of what i did?

https://gist.github.com/scyto/f4624361c4e8c3be2aad9b3f0073c7f9

There is a wealth of information in your git. I'm sorry. I just found what you meant (https://gist.github.com/scyto/7e9d471c70f92c8b983f38cdc838998d) and going to test it directly. You basically created the bible I'm following now.
I'll keep you posted... I hope you still be willing to respond to my stupid questions -;)

heheh, i am still on the fence if i do or don't like the glusterFS volume plugin as it works well until it doesn't - but the fix is easy, just renaable it on each node, also see some of my comments under each articles, even i am not sure if i did things the 'best way

next up is to move all the storage from gluster to ceph

I was just looking at the plugin and got already stuck that somehow I need to use make build command. Not sure what this means ?

Where do you see me saying to make anything in this installing a glusterfs volume plugin · GitHub<https://gist.github.com/scyto/7e9d471c70f92c8b983f38cdc838998d>? Are you reading the plugins git? Don’t, the author of the plugin saying if you want to build the plugin from source how you would do that, given you don’t know the make command, building from source is not something you should be thinking about. I like you want to understand from first principles, but in this case, don’t do that 😊 (at least not yet….) From: Jens-Wymeersch ***@***.***> Date: Thursday, November 14, 2024 at 11:00 AM To: Jens-Wymeersch ***@***.***> Cc: Author ***@***.***>, Comment ***@***.***> Subject: Re: scyto/docker-swarm-portainer.md @Jens-Wymeersch commented on this gist.

…

________________________________ I was just looking at the plugin and got already stuck that somehow I need to use make build command. Not sure what this means ? — Reply to this email directly, view it on GitHub<https://gist.github.com/scyto/a57d63a3b905b24e9eb25618848c8e80#gistcomment-5283329> or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACWXVXGXV4KRSDGBWEKEANL2ATXMVBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTCNBVGEYTMMJWU52HE2LHM5SXFJTDOJSWC5DF>. You are receiving this email because you authored the thread. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

@scyto So I need to run the command on each node in the docker swarm ?
I assume I do but I want to make sure before I screw things up

Just follow the docs I made. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Jens-Wymeersch ***@***.***> Sent: Thursday, November 14, 2024 11:17:53 AM To: Jens-Wymeersch ***@***.***> Cc: Author ***@***.***>; Comment ***@***.***> Subject: Re: scyto/docker-swarm-portainer.md @Jens-Wymeersch commented on this gist.

________________________________ So I need to run the command on each node in the docker swarm ? — Reply to this email directly, view it on GitHub<https://gist.github.com/scyto/a57d63a3b905b24e9eb25618848c8e80#gistcomment-5283390> or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACWXVXFDAVEJI66JPS2RNN32ATZODBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTCNBVGEYTMMJWU52HE2LHM5SXFJTDOJSWC5DF>. You are receiving this email because you authored the thread. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

I looked at the docs, it doesn't specify if you need to run it in the master node or in all nodes.

IF you mounting the filesystem at boot if so you need to make sure fstab doesn't try and mount them utill the network AND the ceph/glsuter services are full up and operational (not just started)

this will require you mess with service dependencies...

consider running a scripted mount command instead of relying on fstab

this was one reason I used the volume driver, can't recall if i modified docker service to start only once gluster was fully up

If thats not the issue maybe you have an inherent networking issue, do all your nodes have names registered in DNS and it might be worth putting all nodes in /etc/hosts - sometimes found gluster would rever to wanting to use nodename for no reason i could fathom

A bit late comment, but by adding the "_netdev" to your mount options in the /etc/fstab file will solve the problem of mounting before the network is online.