You have a Django Project on a Virtualenv but trying to configure Celery to work with Supervisord on CentOS 7 server?. This guide will show you simple steps you need to Configure Celery Supervisord CentOS 7 and Django Virtualenv.
What’s Celery?
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on the real-time operation but supports scheduling as well. It is written in Python, but the protocol can be implemented in any language.
The recommended message broker for Celery is RabbitMQ, but support for Redis, Beanstalk, MongoDB, CouchDB, and databases (using SQLAlchemy or the Django ORM) is also available.
On Celery, the execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
What’s Supervisor?
Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems. Supervisor is configured through a simple INI-style config file that’s easy to learn. It also provides you with one place to start, stop, and monitor your processes which can be controlled independently or in groups.
The server piece of a supervisor is named supervisord. It is responsible for starting child programs at its own invocation, responding to commands from clients, restarting crashed or exited subprocesses, logging its subprocess stdout and stderr output, and generating and handling “events” corresponding to points in subprocess lifetimes.
Install Celery
On your Virtualenv, use pip command to install celery.
$ pip install celery
Install Supervisor on CentOS 7
To install supervisor on CentOS 7, you need epel repository which can be added using the commands.
$ sudo yum -y install epel-release
Then do the installation of a supervisor from epel.
$ sudo yum -y install supervisor
The service can be managed using systemd, the name is supervisord
Manage Celery Programs with Supervisord
Before you can use supervisord to do anything useful, you’ll need to add at least one program section to its configuration. The program section will define a program that is run and managed when you invoke the supervisord command. By default, Supervisor is configured to read all configuration files under /etc/supervisord.d directory. I’ll place my celery configuration file on /etc/supervisord.d/celery.ini instead of modifying supervisord.conf file. Its content looks like below:
# cat /etc/supervisord.d/celery.ini [program:opalquickcelery] command=/srv/venv/bin/celery worker -A opalquick --loglevel=INFO directory=/srv/demo user=shockwave numprocs=1stdout_logfile=/var/log/celery/mail_beat.log stderr_logfile=/var/log/celery/mail_beat.logautostart=true autorestart=truepriority=999
Explanation:
command: This is the command to be executed when you start supervisord daemon
Once all the changes have been made. Then start the supervisord daemon.
# systemctl start supervisord
If you check on its status, you should see the commands specified on our configuration files executed.
# systemctl status supervisord ● supervisord.service - Process Monitoring and Control Daemon Loaded: loaded (/usr/lib/systemd/system/supervisord.service; disabled; vendor preset: disabled) Active: active (running) since Sun 2018-03-18 23:24:09 EAT; 26min ago Process: 1879 ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf (code=exited, status=0/SUCCESS) Main PID: 1882 (supervisord) CGroup: /system.slice/supervisord.service ├─1882 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf ├─1884 /srv/venv/bin/python /srv/venv/bin/celery worker -A opalquick --loglevel=INFO Mar 18 23:24:09 ns2.rafikihost.com systemd[1]: Starting Process Monitoring and Control Daemon... Mar 18 23:24:09 ns2.rafikihost.com systemd[1]: Started Process Monitoring and Control Daemon.
Supervisor also provides a command line tool called supervisorctl which allows you to control the processes that are currently managed by supervisord.
# supervisorctl status opalquickcelery opalquickcelery RUNNING pid 1884, uptime 0:29:24
To stop the process with supervisorctl, use:
# supervisorctl stop <program-name>
To restart, use:
# supervisorctl restart <program-name>
