From the course: Data Ingestion with Python
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Processing event-based data - Python Tutorial
From the course: Data Ingestion with Python
Processing event-based data
- [Instructor] In some cases, instead of pulling data, it will be pushed to us. Digital architecture is to have some kind of a message queue, and clients listening to it. There are many message queue systems: Kafka, RabbitMQ, Amazon SQS, Google Pub/Sub, Azure Queue, and more. The details about choosing each system vary. But the idea is the same. You subscribe to a topic or a subject, and get notified when messages are published to this topic. Most of the time, messages are just a sequence of lines, and you need to decode them with the appropriate serialization format. JSON, MessagePack, Protocol Buffer, et cetera. We are going to use a message queue called NATS. To run the server, you're going to use Docker. If you'd like to know more about Docker, check out our courses on it. run --rm --publish=4222:4222 Publish the Docker port 4222 to the localhost. And we're going to run the NATS server. And it says that the server is ready. A similar technique publishing messages to the right…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.