In many situations, you might want to return a response to the user as quickly as possible, and let the server execute some process asynchronously. Using Celery, you can create asynchronous tasks easily, let them be executed by workers as soon as possible, and schedule them to run at a specific time.
- [Instructor] Hi welcome to the last video of this section. Launching asynchronous tasks with Celery. Previously we saw about registering customer orders. In this video, we will first install Celery install RabbitMQ and add Celery to our project. We will then add asynchronous tasks to our application monitoring Celery. Let's launch asynchronous tasks with Celery. Everything you execute in a view will affect response time. In many situations you might want to return a response to the user as quickly as possible and let the serve execute some process asynchronously.
This is especially relevant for time-consuming processes or processes substitute failure which might need a retry policy. For example a video sharing platform allows users to upload videos but requires a long time to transcode uploaded videos. The site might return a response to the user telling the transcoding will start soon and start transcoding the video asynchronously. Another example is sending emails to users. If your site sends email notifications from a view, the SMTP connection might fail or slow down the response.
Launching asynchronous tasks is essential to avoid blocking execution. Celery is a distributed task queue that can process fast amounts of messages. It does real time processing but also supports task scheduling. Using Celery not only can you create asynchronous tasks easily and let them be executed by workers as soon as possible. But you can also schedule them to run a specific time. You can find this Celery documentation at Celery.readthedocs.org. Let's install Celery and integrate it into our project.
Install Celery via PIP using the pip install celery command. It might take a few seconds. So now Celery is installed. Celery requires a message broker in order to handle requests from an external source. The broker takes care of sending messages to Celery workers which process tasks as they receive them. Let's install a message broker. Now we look into installing RabbitMQ. There were several options to choose the message broker for Celery including key value stores such as Reddit or an actual messaging system such as RabbitMQ.
We will configure Celery with RabbitMQ since its a recommended message worker for Celery. If you're using Linux you can install RabbitMQ from the shell using the command aptget install RabbitMQ. If you need to install RabbitMQ on Mac OS10 or Windows you can find standalone versions at RabbitMQ.com/download.html. After installing it, launch RabbitMQ using the RabbitMQ-server command from the shell. Next we add Celery to your project. You have to provide a configuration for the Celery instance.
Create a new file next to the settings.py file of myshop and name it celery.pi. This file will contain the Celery configuration for your project. Add these lines of code to it and save. In this code, we set the Django settings module variable for the Celery command line program. We then create an instance of our application with the app equals Celery Myshop. We load any custom configuration from our project settings using the config from object method. Finally we tell Celery to also discover asynchronous tasks for the applications listed in the installed apps setting.
Celery will look for a task.py file in each application directory to load asynchronous tasks defined in it. You need to import this Celery module in the init_py file for your project and make sure it is loaded when Django starts. Edit the Init_Py file of my shop and add #importcelery from .celery import app as celery_app. Save the file.
Now you can start programming asynchronous tasks for your applications. The Celery always eager setting allows you to execute tasks locally in a synchronous way instead of sending them to the queue. This is useful for running unit tests or the projects in your local environment without running Celery. Now we create an asynchronous task to send an email notification to our users when they place an order. The convention is to include asynchronous tasks for your application in a tasks module within your application directory. Create a new file inside the oldest application and name it tasks.py.
This is the place where Celery will look for asynchronous tasks. Add these lines of code to it. And save it. We define the order created task by using the task decorator. As you can see a Celery task is just a python function decorated with a task. Our task function receives an order ID parameter. It is always recommended to pass only IDs to task functions and look up objects when the task is executed. We use the send mail function provided by Django to send an email notification to the user to post the order.
If you don't want to set up email settings, you can tell Django to write emails to the console by adding this line to the settings.py file. Use asynchronous tasks not only for time consuming processes but also for other processes that are subjected to failure which do not take so much time to be executed but which was just subject to connection failures or require a retry policy. Now we have to add the task to our order create view. Open the views.py file of the orders application.
And import the task while inserting the import line for order created. We also launch the asynchronous tasks by calling veed-lay method. So call the order created asynchronous task after clearing the cart as we do here. We call the lay method of the task to execute it asynchronously. The tasks we add it to the queue and we execute it by worker as soon as possible. Open another shell and start the Celery worker using the command celery -A myshop worker -l info.
The Celery worker is now running and ready to process tasks. Make sure the django development server is also running. Now open the URL 127001:8000 in your browser. Add some products to your shopping cart and complete an order. So we insert some random values here for name, email address. And click on place order. In the shell, while you started the Celery worker you can see this output. This means the task has been executed and you'll receive an email notification for your order.
You might want to monitor the asynchronous tasks as they're executed. Flower is a web based tool for monitoring Celery. You can install Flower using the command pip install flower. Once installed, you can launch flower by running Celery -A myshop flower from your project directory. Now open your browser and type in the URL localhost:5555/dashboard. You can now see the active Celery workers and asynchronous tasks statistics. You can find documentation for flower at flower.readthedocs.org.
Amazing in this section, we've created a basic shop application. We created a project catalog and built a shopping cart using sections. We also implemented a custom context processor to make the cart available to your template saying created a form for placing orders. Finally we learnt how to launch asynchronous tasks with Celery. In the next section, you'll learn how to integrate a payment gateway into your shop, add custom actions to the administration site, export data as CSV and generate PDF files dynamically.
Note: This course was created by Packt Publishing. We are pleased to host this training in our library.
- Creating an online shop project
- Registering customer orders
- Managing payments and orders
- Integrating a payment gateway
- Extending the online shop
- Creating a coupon system
- Translating Python code