In this post I’ll show you how to setup Supervisor on Linux and become familiar with a few handy options and settings. If you haven’t used Supervisor in the past this post should give you a decent start.
Supervisor is a Python program that makes managing other programs (processes) a breeze. It can ease the process of starting, killing and restarting applications that you develop. I’ll be explaining various aspects that you need to keep in mind while using Supervisor.
- Supervisor can help you administrate and automate the process of starting applications on your web server. It can run the steps specified by you and start multiple applications. Maybe you want somebody to automatically start the application after you’ve restarted your server. Supervisor can do all that and more.
- Supervisor starts your application or script as it’s own child process. This way it can kill them when you want to. It daemonizes something that runs infinitely say a web server. You should not use it to control a process which has a definite end (say a task which takes 3 minutes and exits after that).
- The command to start a particular process and various options are given via a
- Supervisor looks for files ending with
.confin certain directories. It then follows the instructions that you’ve given in those files.
- Once a proper
.conffile is written, starting any program is as simple as
supervisorctl start appname.
- It daemonizes the process that you want to run. You can save logs that your application spits out in files.
- Say your server runs 50 processes. It would be a pain to start every one of them whenever you restart the server. Supervisor has an autostart option which will start the process when Supervisor is initialised.
- You could make Supervisor run a program as a particular user. There are loads of such options that can help make your day to day life easier.
On Ubuntu or debian you can
sudo apt-get install supervisor
- You can also install it using pip. Supervisor on Pypi.
Writing conf Files
Supervisor can be informed of any program that you wish to run via a
I found this example from Digital Ocean to be very helpful. So I’m adapting a similar approach but with a different script. For the sake of this example and brevity we’ll write a small script that displays the amount of RAM that the machine sports.
Go to your home directory (~/) and save the following as
#!/bin/bash counter=0 while true do ((counter++)) echo "Count is now $counter." cat /proc/meminfo | grep MemTotal sleep 1 done
Do note this is a never ending process. It runs forever because of the while loop. I repeat Supervisor can only manage and monitor never ending processes.
Make the file you just created executable.
chmod +x displayram.sh
Save the following as
displayram.conf. Supervisor will use this file to control the process. Substitute abhirath with your username.
[program:displayram] command=/home/abhirath/displayram.sh autostart=false autorestart=false stderr_logfile=NONE stdout_logfile=/home/abhirath/displayram.out.log
Move the file that you just created to
/etc/supervisor/conf.d/displayram.conf. This is the default directory that Supervisor looks for programs.
Then on the terminal
supervisorctl start displayram
Our script starts instantly. You can verify that it works by opening
/home/yourusername/displayram.out.log. It should be filled with lines displaying the RAM installed on the machine.
A better way would be to run the following on the terminal.
tail -f /home/yourusername/displayram.out.log # or tail -f ~/displayram.out.log
It will keep printing out new entries on the terminal.
You will also need to familiarise yourself to spawn more than one process - An example which requires running multiple processes to start an app.
To stop the script we just started run
supervisorctl stop displayram
A Few Options Explained
- The directory which Supervisor switches to before executing the command.
- Researching about this setting to get my app to work with Supervisor was when I learnt that when you activate an environment in Python you basically are just instructing your machine to use a copy of Python situated in a particular directory instead of using the default Python installation.
- The way via which Supervisor starts a particular program. Give the command / path that you normally use to start your program.
- Commands are not equivalent to bash or your typical terminal commands by default. They are fairly similar but a command that you use on your terminal may not yield the same effect here.
- Commands can contain absolute paths or relative paths. The
displayramexample uses absolute paths. If you want to use relative paths you need to set the path. You can read more about it here.
- You can ensure that your command is treated as a bash command by using
bash -c "command here". Our earlier example can be rewritten as
[program=displayram] directory=/home/abhirath command=bash -c "./displayram" autostart=false autorestart=false stderr_logfile=NONE stdout_logfile=/home/abhirath/displayram.out.log
Stop and Kill as Group
- Sometimes your program doesn’t really get killed. They might have children that they don’t kill. So to actually stop them Supervisor has to be mean and kill every process that might have spawned. One possible reason is that they expect a different kill signal.
- You don’t have to use them unless you notice that your program is still running even after you explicitly ran
supervisorctl stop programname
- This helped me a lot - Really killing a program.
stderr_logfile- the file to which it shall redirect stderr to. If set to NONE it will not log output from stderr.
stdout_logfile- the file to which it redirect stdout to. This can also be set to NONE.
stderr_logfile_maxbytes- maximum space that the log file will take before Supervisor uses a new file to store logs. You can input a numerical followed by GB, KB etc. By default it’s set to 50MB.
stdout_logfile_maxbytes- same as above but for stdout.
stdout_logfile_backups- the number of backup files it stores. Say you’ve set it to 3. After the first files exceeds the value of
stdout_logfile_maxbytesit creates a second file then a third. If the third file exceeds the size it proceeds to delete the first file and continue logging. In all it would use a maximum of 3 * 50MB in our case. By default it has a value of 10.
stderr_logfile_backups- same as above but for stderr.
- An example which uses a variety of log settings.
Supervisor saves it’s own logs as well. They are different from the logs you set for your program. These logs contain information about Supervisor. They are stored in
To see the last few logs you can
A useful read - Options available under program.
Some Common Errors
1. You get a message similar to
unix:///var/run...sock no such file?
It means Supervisor isn’t running. You will have to start Supervisor.
sudo service supervisor restart
2. Spawn errors. Supervisord is unable to start your program?
- Check the error log and output logs. Supervisor keeps appending stuff to your program’s log file. So it’s a good idea to use
tail -f /path/to/program/log/fileto keep track of changes happening.
- Check the
directoryoptions under your program. Give absolute paths like
3. Supervisor doesn’t start automatically after bootup?
- On Ubuntu 16 you can enable it by
sudo systemctl enable supervisor. Also read this stackexchange question. - init scripts for various operating systems.
- Frequently new
.conffiles that you write are not recognised. If that is the case do the following:- Check the syntax of everything in the file. If there’s an error Supervisor doesn’t even show the file. Also the files must have an extension of
4. Any signs of the process / app that you want to start is already running?
- They might already be running. Check the status using
supervisorctl appname status.
- I’ve seen .NET apps that don’t kill their child processes by themselves. So even after you stop a program their child process might be running. For instance you might get errors like port is already in use.
- Read about
killasgroupwork on individual programs. They are not to be confused with group applications
- Really killing a program.