Start learning with our library of video tutorials taught by experts. Get started

Node.js Essential Training
Illustration by Richard Downs

Streaming chunked data through readable streams


From:

Node.js Essential Training

with Joseph LeBlanc

Video: Streaming chunked data through readable streams

Reading an entire file into memory can be troublesome, especially if the file is several gigabytes in size. A better way of handling large files is to stream the data instead. In this video we'll stream the data from a file and read individual chunks. Let's go to the exercise files. Open up chapter nine and video two, and then copy those two files to the Desktop. Now let's go to Terminal and change directory to the Desktop. It's capital D, and then press Return.
Expand all | Collapse all
  1. 3m 9s
    1. Welcome
      42s
    2. What you should know before watching this course
      57s
    3. Using the exercise files
      47s
    4. Using the challenges
      43s
  2. 18m 50s
    1. Why use Node.js?
      2m 1s
    2. Choosing a Node.js installation process
      4m 56s
    3. Installing Node.js on the Mac
      2m 5s
    4. Installing Node.js on Windows
      1m 43s
    5. Installing Node.js via Node Version Manager on Ubuntu Linux
      4m 45s
    6. Installing Node.js via Node Version Manager on a Mac
      3m 20s
  3. 13m 54s
    1. Exploring language additions to the V8 JavaScript engine
      3m 38s
    2. Understanding require() and modules
      6m 39s
    3. Understanding callback execution in the event loop
      3m 37s
  4. 12m 23s
    1. Initializing Node.js projects
      2m 37s
    2. Finding an NPM
      3m 30s
    3. Maintaining projects using the npm command
      6m 16s
  5. 21m 4s
    1. Creating modules with getters and setters
      6m 33s
    2. Understanding module caching and scopes
      6m 15s
    3. Implementing JavaScript creational patterns
      5m 8s
    4. Challenge: Leveraging module caching
      55s
    5. Solution: Leveraging module caching
      2m 13s
  6. 32m 27s
    1. Starting Express applications
      3m 42s
    2. Handling GET requests returning JSON
      3m 26s
    3. Navigating large datasets using request variables in routes
      5m 4s
    4. Modifying data through PUT requests
      4m 32s
    5. Supplying middleware to modify HTTP response headers
      4m 37s
    6. Generating HTML views in Jade
      8m 9s
    7. Challenge: List all records as JSON
      37s
    8. Solution: List all records as JSON
      2m 20s
  7. 25m 37s
    1. Evaluating unit testing methodologies
      3m 8s
    2. Isolating existing code for testing
      6m 47s
    3. Supplying helpers for test data
      2m 35s
    4. Writing unit tests with Mocha
      6m 5s
    5. Simulating HTTP requests with SuperTest
      7m 2s
  8. 28m 45s
    1. Connecting to MongoDB via Mongoose
      4m 16s
    2. Defining schemas for collections in Mongoose
      3m 47s
    3. Querying MongoDB collections
      5m 32s
    4. Persisting Express sessions through MongoDB and mongo-connect
      7m 6s
    5. Logging in to Express applications using Passport
      8m 4s
  9. 21m 47s
    1. Controlling the built-in REPL
      4m 18s
    2. Constructing a custom REPL
      5m 40s
    3. Receiving command-line arguments with Optimist
      6m 14s
    4. Building command-line tools
      3m 35s
    5. Challenge: Launch a custom REPL from a command
      41s
    6. Solution: Launch a custom REPL from a command
      1m 19s
  10. 15m 46s
    1. Emitting events and attaching listeners
      5m 32s
    2. Streaming chunked data through readable streams
      2m 33s
    3. Controlling readable streams
      1m 52s
    4. Piping readable data into writable streams
      2m 10s
    5. Handling duplex streams over TCP
      3m 39s
  11. 4m 6s
    1. Deploying projects to cloud hosting services
      2m 36s
    2. Finding Node.js resources
      1m 30s

Watch this entire course now—plus get access to every course in the library. Each course includes high-quality videos taught by expert instructors.

Become a member
Please wait...
Node.js Essential Training
3h 17m Intermediate Oct 16, 2013

Viewers: in countries Watching now:

Node.js is a powerful tool for controlling web servers, building applications, and creating event-driven programming. And it brings JavaScript—a language familiar to all web developers—into an environment independent of web browsers. Learn all about Node.js and start creating JavaScript applications in this course with Joseph LeBlanc. He shows how to install Node.js on Mac, Windows, and Linux and dives deep into its modules and Express framework for app development. Learn how to unit test your code, start sessions on web servers, stream data, and create simple command-line tools.

Topics include:
  • Why use Node.js?
  • Installing Node.js
  • Understanding the event loop
  • Initializing Node.js projects
  • Creating modules with getters and setters
  • Starting Express applications
  • Testing your code
  • Working with sessions and databases
  • Building command-line tools
  • Emitting events and attaching listeners
  • Controlling readable streams
Subjects:
Developer Servers Cloud Computing Programming Languages
Software:
JavaScript Node.js
Author:
Joseph LeBlanc

Streaming chunked data through readable streams

Reading an entire file into memory can be troublesome, especially if the file is several gigabytes in size. A better way of handling large files is to stream the data instead. In this video we'll stream the data from a file and read individual chunks. Let's go to the exercise files. Open up chapter nine and video two, and then copy those two files to the Desktop. Now let's go to Terminal and change directory to the Desktop. It's capital D, and then press Return.

Now let's take a look at these files. I'm going to take a look at stream.js and pull it into sublime text. On line one, I'm requiring the FS Module. The FS module is a core module that deals with the file system. Then on line three, I'm calling the create read stream method of FS. I'm creating a stream out of data.json. Instead of reading all the contents of the file at once and storing it in the stream, it's creating a stream so that we can read the data one chuck at a time.

Then on line five and on line 12, I'm defining two different listeners to the data event. These two listeners are functions. These functions accept a chunk as the single argument. In this listener I'm marking the beginning and the end of each chunk, and then in the middle, I'm converting the chunk to a string. In the second listener, I'm simply outputting the length of the chunk. There's one more listener in this file. It's listening for the end event. This event is going to get fired when the stream reaches its end.

And then all I'm doing in this listener is outputting that we've reach the end of the file. So let's go back to Terminal and run this file. I'm going to type node and stream.js. Now I'm going to scroll all the way back up to where this started. Now in this case, our file is pretty small, and the string finished in one chunk. However, both data listeners fired. The chunk was output to the console, and we also displayed the length of the data in the chunk.

If this were a much larger file, this would be a more efficient way of reading the data than trying to wait for the entire file to load in memory. Streaming a file makes it possible to read data before the entire file has loaded. One or more listeners can be attached to a file stream to handle chunks, and end listeners can perform any finishing cleanup. In the next video, we'll have a look at how streams can be paused.

There are currently no FAQs about Node.js Essential Training.

Share a link to this course
Please wait... Please wait...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.
Upgrade now


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

join now

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Node.js Essential Training.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Welcome to the redesigned course page.

We’ve moved some things around, and now you can



Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked