navigate site menu

Start learning with our library of video tutorials taught by experts. Get started

Up and Running with NoSQL Databases

Up and Running with NoSQL Databases

with Joseph LeBlanc

 


Explore the possibilities NoSQL databases offer developers for unparalleled flexibility and performance. In this course, author Joseph LeBlanc shows how to build an entire web application with CouchDB and introduces techniques such as querying with JavaScript, working with nested data, and creating views. He also explains some of the trade-offs you might encounter with NoSQL databases, such as the complexity of partitioning data across multiple servers, and presents five commonly used NoSQL databases: CouchDB, MongoDB, Cassandra, Riak, and Redis.
Topics include:
  • What is NoSQL?
  • Installing CouchDB on Mac, Windows, and Linux
  • Storing data
  • Nesting document data
  • Specifying search criteria
  • Reducing data
  • Attaching and retrieving images
  • Deploying applications
  • Securing CouchDB
  • Understanding the CAP theorem

show more

author
Joseph LeBlanc
subject
Developer, Databases
software
NoSQL , CouchDB , MongoDB , Cassandra , Riak , Redis
level
Intermediate
duration
56m 57s
released
Nov 30, 2012

Share this course

Ready to join? get started


Keep up with news, tips, and latest courses.

submit Course details submit clicked more info

Please wait...

Search the closed captioning text for this course by entering the keyword you’d like to search, or browse the closed captioning text by selecting the chapter name below and choosing the video title you’d like to review.



Introduction
Welcome
00:04Welcome to Up and Running with NoSQL Databases.
00:07I'm on Joseph LeBlanc, and in this course we'll be looking at storing schema free data
00:12and querying against different fields,
00:15serving complete web applications from CouchDB based on JavaScript and HTML,
00:22comparing different NoSQL databases and understanding the trade-offs of each.
00:28NoSQL databases offer unprecedented flexibility and performance tuning options.
00:34So let's get Up and Running with NoSQL Databases.
Collapse this transcript
What you should know before watching this course
00:00There are a few things you will want to be familiar with before learning about NoSQL databases.
00:05First, you will want to be familiar with queries.
00:08Queries are used to read data from the database and write it back to the database.
00:14Next, you'll want to understand indexes and keys.
00:18Indexes and keys are essential for tuning the performance of a database.
00:22You'll also want to be familiar with JavaScript.
00:25JavaScript is used in many NoSQL databases as the querying language for the database.
00:31And finally, it's also helpful to be familiar with JSON.
00:34JSON is frequently used as a storage format within NoSQL database records.
00:40With these concepts in mind, let's dig into NoSQL databases.
Collapse this transcript
Using the exercise files
00:00Some of the videos in this course, will be using CouchDB as a way of visualizing NoSQL concepts.
00:06These videos include exercise files.
00:09The exercise files are organized by chapter with each chapter including folders for each video.
00:16Each video's folder contains at least one file with a .couch extension.
00:21The folder sometimes also includes other files that are used within the video.
00:26If you want to get your database to the starting point where I start the video, use the file
00:30with the .couch extension.
00:33You'll want to copy this file to a specific path.
00:36This path varies depending on which operating system you're using.
00:40The paths displayed here are the standard ones for each operating system.
00:44Before you copy the file with a .couch extension to one of these paths, be sure to stop CouchDB,
00:50replace the file, and then restart CouchDB.
00:54In Chapter 2, I'll be showing you how to install CouchDB on Mac, Windows and Ubuntu Linux.
01:00
Collapse this transcript
1. Overview
What is NoSQL?
00:00NoSQL has become a bit of a buzzword in recent years.
00:04Some claim that using NoSQL databases can solve all of your website scalability issues.
00:09But is this really true?
00:11And what is a NoSQL database anyway? A NoSQL database is simply a database that does not use SQL.
00:20SQL was designed to be a query language for relational databases,
00:24and relational databases are usually table- based, much like what you see in a spreadsheet.
00:29In a relational database, records are stored in rows and then the columns represent fields
00:34in each row. Then SQL allows you to query within and between tables in that relational database.
00:41On the other hand, NoSQL databases are more flexible.
00:45Many NoSQL databases allow you to define fields as you create a record.
00:51Nested values are common in NoSQL databases. You can have hashes and arrays and objects,
00:57and then nest more objects and arrays and hashes within those.
01:02And then that way you can have nested values in your NoSQL database records.
01:08Also fields are not standardized between records in NoSQL databases.
01:13You can have a different structure for every record in your NoSQL database.
01:17While NoSQL is being promoted as a recent phenomenon, NoSQL is not new.
01:24The Apollo space program used a NoSQL database to catalog the inventory for the parts of
01:29the space shuttle.
01:31Many embedded devices also use NoSQL databases, and many of the early mainframe computers
01:37use NoSQL databases as well.
01:40Just like relational databases, NoSQL databases have trade-offs and limitations.
01:47NoSQL databases will not solve your website scalability issues.
01:51You have to weigh the benefits and drawbacks of each database and see if they fit your
01:56application needs.
01:58On the other hand, NoSQL databases offer flexibility that's unavailable through relational databases.
02:05The main characteristic separating NoSQL databases from relational SQL databases is that they
02:11do not use query languages derived from SQL.
02:15Additionally, many NoSQL databases are flexible in ways that relational databases are not.
02:22In the next few videos, we'll take a look at the different types of NoSQL databases
02:26and what they offer.
Collapse this transcript
Defining NoSQL database types
00:00There are numerous types of NoSQL databases.
00:03Not all NoSQL databases operate under the same principles, but many of them fall under
00:08one of many broad categories.
00:10First, they are document stores.
00:13Documents are usually stored in a structured format such as XML or JSON or some other standardized
00:19format that can be queried.
00:22Documents are usually organized into collections or databases.
00:26This allows you to group similar documents all together.
00:29Individual documents can have unique structures. You don't have to have the same fields in each document.
00:36Each document usually has a specific key. This allows you to retrieve the document quickly,
00:42and normally it's possible to query a document by the specific fields contained inside.
00:48Next, there are key-value stores. Typically you have a key you can query by, and then
00:53there's a specific value at that key.
00:56However, you usually can't query by anything other than the key.
01:01Some key-value stores let you define more than one key though, which can help you get
01:05around the drawback.
01:07Key-value stores are sometimes used alongside relational databases for caching.
01:12Next we have BigTable/tabular databases.
01:15Tabular databases are named after Google's proprietary BigTable implementation.
01:21Each row in a tabular database can have a different set of columns.
01:25Tabular databases are designed for large numbers of columns in each row, and rows are typically versioned.
01:32They're also graph databases.
01:34Graph databases are designed for data that's best represented as interconnected nodes.
01:39So for example, if you have a series of rowed intersections that you want to keep in a database,
01:44a graph database is a good way of doing that.
01:48And then finally, there are object databases.
01:51Object databases are tightly integrated with object-oriented programming languages.
01:56Object databases act as a persistence layer.
01:58You store the objects from your programming language directly into the object database.
02:04You can also link objects together through pointers.
02:07In this course, we'll be exploring a document database.
02:11While document databases do not necessarily enforce a specific data structure, they typically
02:16give the capability of querying and indexing against fields within each document.
Collapse this transcript
Exploring NoSQL possibilities
00:00You may be wondering when you should use NoSQL database instead of an SQL-based relational one.
00:06Let's take look at some examples of instances where NoSQL databases give you an advantage.
00:12Many NoSQL databases can be used to easily create web applications with customizable fields.
00:17You don't have to worry about schema changes and you can still index the fields.
00:22For instance, if you have an application that keeps track of comments on articles, you can add votes later.
00:28Another possibility for NoSQL is to use it as a caching layer.
00:32You might have a relational database that you want to keep traffic away from.
00:37You can use NoSQL as a caching layer and write code to pull data from the relational database
00:42and store it in NoSQL, then you can use NoSQL for your API.
00:47Many NoSQL databases can also store binary files.
00:50With NoSQL you don't have to worry about file system permissions.
00:55NoSQL databases will frequently extract all the metadata about a file and allow you to query by it.
01:01You can also use NoSQL to attach other data to specific files.
01:06And finally, some NoSQL databases allow you to serve full web applications.
01:11Your HTML, stylesheets, and JavaScript can be served directly from NoSQL, then you can
01:17use the permissions in NoSQL to control who can read and write data.
01:22NoSQL databases can be used to build new applications from scratch as well as augment the capabilities
01:27of a relational database.
01:30Even if you're working with an existing application, you can start using NoSQL databases now to
01:35take advantage of their features.
Collapse this transcript
2. Accessing NoSQL Databases
Access overview
00:00This course uses CouchDB to explore concepts common to many NoSQL databases.
00:06CouchDB is available as a download that you can run on your local computer.
00:11When you run CouchDB on your local computer, the Futon user interface comes up automatically.
00:17If you are not permitted or you don't want to install CouchDB on your local computer,
00:23you can also use a cloud service to run CouchDB.
00:28The Futon user interface for both options is the same.
00:32However, you may have difficulty using some of the exercise files with the cloud service.
00:39Installing CouchDB on your computer is the best way of experiencing this course.
00:43However if installing it locally is not an option, cloud-based services are also available.
00:49
Collapse this transcript
Installing CouchDB on Windows
00:00To install CouchDB on your Windows computer, download it from couchdb.apache.org. Click
00:06the download link.
00:09I am selecting the version that works with Erlang R15B. Clicking on the download link
00:16takes you to a mirror selection page. I'm going to select the default mirror.
00:24When the download is complete, double-click to start the installation.
00:31First, accept the license. I'm going to leave this path as the default.
00:37You can also create a Start Menu folder.
00:41If you don't want CouchDB to start up whenever you start Windows, uncheck these boxes, otherwise
00:47leave them checked. Finally, click the Install button.
00:52Uncheck the View README.txt box and then click Finish.
00:57CouchDB is now installed on your Windows computer.
01:00Let's fire up the Futon interface.
01:03Click the Start button and go to All Programs, then find Apache CouchDB. Click the Futon link.
01:12Now let's have a look at the folder to use when using the database dumps from the exercise files.
01:17Open up Explorer, and then go to C: > Program Files (x86) > Apache Software Foundation > CouchDB,
01:28var > lib > couchdb.
01:32This is the folder to use to copy in database dumps from the exercise files.
01:37Now you have everything you need to use CouchDB on your Windows computer.
Collapse this transcript
Installing CouchDB on Macs
00:00To install CouchDB on your Mac, download it from couchdb.apache.org.
00:07Click on the download link.
00:12Once the ZIP file finishes downloading, double-click on it.
00:19This extracts the CouchDB application to your Downloads folder.
00:23Now, you can just drag the CouchDB application to your Applications folder.
00:28Navigate to Applications, and then find CouchDB Server.
00:33When you double-click on CouchDB Server, it starts up CouchDB and then opens the Futon user interface.
00:41You'll also notice that there is a quick icon in the upper right-hand corner of your screen.
00:46This allows you to quit CouchDB or open Futon again.
00:50If you want to use the database dumps that come with the exercise files, you'll need
00:54to use a specific path to load them in.
00:57Go to Finder and click Go from the menu.
01:00Hold down the Option key and Library appears.
01:06From here, navigate to Application Support > CouchDB > var > lib > couchdb.
01:14This is where you can paste in the database dump files.
01:17Now you have everything you need to use CouchDB on your Mac computer.
Collapse this transcript
Installing CouchDB on Linux
00:00To install CouchDB on Ubuntu Linux, you can either use the apt-get package manager or
00:06you can build it from source.
00:08We'll use the apt-get package manager.
00:12Go to Terminal and then type in sudo apt-get install couchdb.
00:23If it prompts you to install any dependencies, say Yes.
00:27CouchDB is now installed and running on our Ubuntu system.
00:34If you go to 127.0.0.1:5984 and then do /_utils, the Futon interface appears.
00:50If you want to use the database dumps that come with the exercise files, you'll need
00:54to load them in at a specific path.
00:57This path is protected, so you'll need to change user to root.
01:03Go to /var/lib/couchdb/1.2.0.
01:12This folder contains the .couch files that make up each database in your Couch server.
01:18You now have everything you need to run CouchDB on Ubuntu Linux.
Collapse this transcript
Signing up for CouchDB in the cloud
00:00Cloud-based CouchDB is a good solution if you do not want or are not permitted to install
00:05software on your computer.
00:07Many cloud-based CouchDB offerings are available and Iris Couch is one of them.
00:12We're going to sign up for CouchDB on Iris Couch now.
00:19After you click Send, it will tell you that your CouchDB instance is ready.
00:26When you click the link to your database, it brings up the Futon admin interface.
00:30Unlike installing CouchDB locally, there's no place to load in database dump files.
00:35Now, you can continue the course using Iris Couch.
00:39Since you cannot load the database dumps into Iris Couch, you may need to re-create some
00:43of the datasets by hand.
00:45You can also follow along with your own data.
00:47
Collapse this transcript
3. Querying Schemaless Data
Storing data
00:00Many NoSQL databases store data without a predefined schema, and CouchDB is no exception.
00:07In this video, we'll start up CouchDB and start storing some data.
00:11First, go to your Applications folder, and navigate to CouchDB.
00:17When you double-click on CouchDB, it automatically opens up the Futon interface.
00:22We already have a couple of databases in this fresh server.
00:26We have _replicator and _users.
00:29These two databases are used to help run CouchDB.
00:31CouchDB uses these databases as a way of organizing documents in specific sets.
00:41Databases are also used to enforce security and access control across all of the documents
00:45that are part of that database.
00:48In this course, we'll be creating a website for a restaurant.
00:51So let's create a database to support that.
00:54Click the Create Database link and then give the database a name of restaurant.
01:01Then click Create.
01:02Now, we have a fresh database with no documents in it.
01:06Let's create the first document by clicking New Document.
01:09You'll notice that it automatically gave us a hash for the ID field.
01:14Under normal circumstances, you'll want to accept this hash value unless you have a good
01:18reason not to use it.
01:20Now, let's start filling in this document.
01:22I'm going to go to the exercise files, and then open content.txt.
01:30So, let's create a welcome document now.
01:34I'm copying Welcome, and now I'm going to click Add Field.
01:38I'm going to give this field a name of name, and the value is going to be Welcome.
01:44I now also want to create another field for the content.
01:48Let's copy The Corner Cafe, and now click Add Field again, and give this field the name
01:54of content, and then paste in The Corner Cafe.
02:00Once these values are in the document, click the green check next to each value.
02:06Now, finally click Save Document.
02:09You'll notice that the _rev field got added.
02:12This keeps track of the revision number.
02:15Each time we make a change to this document, this revision number is going to increment.
02:20Let's create another document in this database.
02:23Click on restaurant, and click on New Document.
02:30We're going to call this document Hours.
02:32So click Add Field and then type in name, add another field and call it content.
02:46And now copy in the hours.
02:50Click the green check next to each field, and then click Save Document.
02:55Now I'm going to add one more field.
02:57I'm going to call this author.
03:00Then, I'm going to add my name here. Click the green check, and then click Save Document.
03:08Now, let's click restaurant, and then open these two documents in new tabs.
03:15You'll notice that we have this content here for The Corner Cafe and the name is Welcome,
03:22while we have author here as another field in addition to content and name.
03:27Even though these two documents are in the same database, they have different field sets.
03:31And this is okay because that's something that CouchDB allows.
03:36Storing data in CouchDB is as easy as naming a database and naming field-value pairs.
03:41The structure does not have to be the same from document to document.
03:45You can add the exact fields you want for each one.
Collapse this transcript
Nesting document data
00:00Flat fields are just one of the many options you have for storing data in CouchDB.
00:05Arrays and objects allow you to nest data within documents.
00:09Right now, we have the Futon interface loaded with the restaurant database in place.
00:14So let's navigate to restaurant.
00:17We already have a couple of documents ready to go.
00:20Click on the one with the second revision.
00:22Here we have the document with the hours.
00:26Let's say we want to add the hours as an object where we can more readily access that data
00:31in a structured format.
00:33Let's go to the exercise files to paste that in.
00:43Copy this object, and then create a new field for it by clicking Add Field.
00:49I'm going to call this field Hours, and now I am going to paste that data into this field,
00:56and then click the green check.
00:58You'll notice now we have each day of the week bolded, and the value is in quotes.
01:04If we click over to the source, you'll see that hours is structured within this document.
01:11One thing that's very important to remember is that the JSON format is very strictly adhered to.
01:17If we edit this field and change to single quotes, the field is then just interpreted as plain text.
01:25Then we lose the structure of this field.
01:34When we change it back to proper JSON, the structure returns.
01:39Let's click on Save Document, and then add a new document.
01:45Navigate to restaurant and click New Document.
01:48We're now going to create a document about the delivery options.
01:54Copy the word Delivery, and then click Add Field, and then add the name name as a field.
02:02Paste in Delivery, and then click Add Field again, and now copy the content.
02:08Name this field content and now paste in the content.
02:15Finally, we're going to add the phone numbers as an array.
02:22This business has two phone lines.
02:24We're only advertising the one phone line for the moment,
02:27but at some point in the future, we might have an application that uses both.
02:31So copy the array, click Add Field, and then type numbers as the field name.
02:42Then paste in the array.
02:44Finally, check off all four fields.
02:50You'll notice that we used proper JSON and the field was interpreted as an array.
02:55Now that we have all the data entered into this document, be sure to save it.
03:00JSON makes it possible for you to nest data in CouchDB as well as other NoSQL databases.
03:07Just make sure the syntax is correct, so that your data is recognized as JSON.
Collapse this transcript
Retrieving data
00:00While the user interface of CouchDB is handy for creating new databases and documents quickly,
00:05you will eventually want to get the data from those documents in a standardized way.
00:10Fortunately, JSON-formatted data is readily available from CouchDB through HTTP.
00:17We've started up CouchDB, and it lists all the databases here.
00:21Let's open up a new tab and go to /_all_dbs.
00:27You'll notice that those three databases are now listed again only in JSON format.
00:35If we go to /restaurant, you might expect that all the records from restaurant are listed.
00:42However, this only lists the information about the restaurant database.
00:46To get a list of all of the documents in the restaurant database, go to /restaurant/_all_docs.
00:56Now again, we see all the same information that you would see if you navigate to restaurant here in Futon.
01:04We'd also like to be able to get the information about a single document.
01:08To do that, copy the ID and paste it in as the last segment of the URL.
01:13All of the data from this document is now displayed in JSON format, including the name
01:21and the content that we entered earlier.
01:23Retrieving documents from CouchDB is as easy as making an HTTP request.
01:27Once you have the key for the document and the database where it's stored, you have everything
01:31you need to get the data back in JSON format.
Collapse this transcript
Specifying search criteria
00:00If you're already familiar with SQL, you might be wondering exactly how you can retrieve
00:05specific data without it.
00:07CouchDB offers querying via JavaScript.
00:11Let's navigate to the restaurant database and then select Temporary View from the View dropdown.
00:17This gives us a screen with a Map Function on the left and an optional Reduce Function on the right.
00:23For now, we're going to concentrate on the Map Function.
00:26There is also a slider here which I'm going to pull down to give us a little bit more room.
00:31The example Map Function that it gives us is one that simply calls emit.
00:36Emit takes two arguments; first the key and then the value.
00:42If we click the default function, it gives us the results; and each key is null, and every
00:49value is the complete document.
00:51Let's specify a key.
00:53Select null and then replace it with doc.name.
00:58When we click Run again, it shows the name of every document as the key and then uses
01:03the entire document as the value.
01:05Let's specify the value.
01:08Go to the second argument and add .content.
01:11Then click Run again.
01:14Now you'll notice this view has the name as the key and the content as the value.
01:20Now let's say we wanted this view to include every document in the database except the
01:26one with the Hours property.
01:28We can wrap this emit function in an if statement.
01:32So I'm typing in if(!doc.hours) and then wrapping the emit statement in this block.
01:42When I click Run again, it shows every document except the one with the Hours property.
01:47To get filtered data from CouchDB, use JavaScript to write a Map Function.
01:52This makes it possible to get a result set with only the data you need.
Collapse this transcript
Defining views
00:00Once you have a Map Function returning the data you want from CouchDB, you can save it as a view.
00:06Let's create a view on this dataset so we can use it again later.
00:09Navigate to the restaurant database.
00:12If it's not already selected, select Temporary View from the dropdown.
00:18It saved the Map Function from our previous Temporary View.
00:21If you don't already have this code in place, copy it in and click Run.
00:27We want to save this view, so we have all the content except for the page with the hours.
00:33Click the Save As button, and then type in the name of the design document and the view name.
00:38The design document is something we can use to build an application with later.
00:43I'm going to name this design document pages and I'm going to name this view without_hours.
00:52So now you'll notice that the View dropdown has pages, our design document, and without_hours,
00:59the name of the view.
01:02We can go to all documents or we can switch to our view.
01:06We can also get to a JSON- formatted version of this data.
01:10Remove everything from utils to the question mark, and then hit Enter.
01:18So now we have the ID for each document as well as the key and the value.
01:24We can also filter this view.
01:26At the end of the URL, add a question mark, type in key, type equals, and then within
01:32quotes, type Delivery.
01:37Now the view is only showing the document that matches the key delivery.
01:42Views provide a permanent place where Map Functions can be stored and run.
01:45With a View function in place, not only do you gain an endpoint for using the data in
01:49your application, you're also able to further filter it by supplying a key.
Collapse this transcript
Reducing data
00:00Sometimes you just want a summary of a particular dataset.
00:03Many NoSQL databases, including CouchDB, allow you to define a reduce function to do this
00:10on the server side.
00:12Let's navigate to the restaurant database.
00:15If you're not already at the All Documents view, go there.
00:19I've already loaded in several documents that we're going to apply a Map and a Reduce function to.
00:25These documents each represent an item that's going to be on the menu for the restaurant.
00:32Each document has the name of the menu item as well as the category that that menu item belongs to.
00:39What I'd like to do with this data is create a reduce function to tell me how many items
00:45are in each category.
00:47Let's do this by going to the Temporary View.
00:50If you already have a function defined here for Map Function, you can just paste over it.
01:03Now I am going to paste in the function and click Run.
01:08This Map Function is detecting whether or not the category property is set on the document.
01:13We're not going to get the other documents that are pages in the system, we just want
01:18the menu items that have the category property.
01:20Next, we emit the category as the key, and the document as the value.
01:25You'll notice that we have three documents that each have the key entree.
01:29This is important because the reduce function is going to be called once for entree, once
01:34for dessert, once for beverages, and once for side dish even though entree is listed
01:40three times. The reduce function is going to receive all three of these documents in an array.
01:46Now let's paste in a reduce function that will allow us to count the items in each category.
01:56Paste the function in.
01:57There's a small quirk in the Futon interface that doesn't allow me to run the reduce function
02:02until I have both the map function and the reduce function before I hit run.
02:06So, click the Refresh button, and then paste in that reduce function and click Run.
02:12Now, click Refresh again, and then click Run again.
02:18Now, you'll notice a Reduce check box.
02:23Click the check box and the reduce function runs.
02:25So the reduce function takes three arguments: key, values, and rereduce.
02:31The reduce function gets called once for each key that the map function supplies.
02:36So even though three documents had the key of entree, the reduce function was called
02:41only once for entree.
02:44The values argument is an array that contains all the values that match up to that key.
02:49So, in this case, the values array had three documents for the key entree.
02:54Finally, we are returning the value that we want for each key.
02:59In this case, we're just returning the length of the array.
03:02You can use whatever logic you want here to return sums or other statistics on your values,
03:09but at the end, you just want to return one value.
03:12Also, whenever you return more than one value, the reduce function is called recursively
03:17and rereduce is set to true.
03:20This can help you reduce complex nested datasets.
03:24Reduce functions work together with map functions.
03:27The map function first retrieves a keyed set of data.
03:30Then, the reduce function takes the values mapped to each key, and transforms them into a single value.
03:36Using this combination allows you to retrieve summaries directly from the database, saving
03:41time and bandwidth.
Collapse this transcript
4. CouchDB Applications
Attaching and retrieving images
00:00While CouchDB makes it possible to store and query JSON-encoded data, it also allows you
00:05to store and retrieve binary files.
00:08CouchDB supports binary data through a feature called Attachments.
00:12Let's add some photos as attachments.
00:15Navigate to the restaurant database.
00:17We have six records at the bottom here that are menu items.
00:21We're going to want to attach photos to all of these menu items.
00:28The first menu item is the key lime pie.
00:30Click the Upload Attachment button, and then click Browse.
00:36Once you've selected the photo of the key lime pie, click the Upload button.
00:40The key lime pie photo now appears in the Attachments field.
00:45The name of the file and the file size, and the file type are displayed.
00:48Now, click on the file name.
00:51This takes us directly to the image.
00:54Notice that the URL is made up of the database name, the ID of the document, and the name of the file.
01:01I am going to close this tab now.
01:03This takes us to the stir fry document.
01:05Click the Upload Attachment button again, and then click Browse.
01:08Click the stir fry photo, and then click Upload.
01:13Again, it's uploaded the stir fry photo to the correct document.
01:18You can also upload multiple files to the same document.
01:22Click the Upload Attachment button again and then click Browse.
01:25I am going to select greenbeans.jpg and click Upload.
01:31You'll notice we now have greenbeans and stirfry as attachments of this document.
01:36To remove greenbeans, click the X button.
01:38Before moving away, click the Save Document button.
01:41This will record the deletion of the greenbeans.jpg photo.
01:45You can continue by adding the remaining photos to the appropriate menu items.
01:50CouchDB makes it possible for you to store binary files along with your schema-less data.
01:56Once you know how the URLs are structured, files can be retrieved through HTTP.
02:01
Collapse this transcript
Querying attachments
00:00Once attachments have been uploaded to your CouchDB database, you'll want to be able to reference them easily.
00:07Let's create a view to help us get all of the attachment names.
00:10First, let's navigate to the restaurant database.
00:12Then head over to the View dropdown, and select Temporary View.
00:17If you have the default function that just emits null on the document, click Run.
00:22Now, let's narrow that down to the Attachments field, add in a dot, and then underscore,
00:27and then attachments.
00:31You'll notice that some of our documents don't have an attachments field, while the other ones do.
00:36We want to have a function that only returns the documents that have attachments.
00:41So let's go to the exercise files and get one.
00:46Open up map.js and copy the function.
00:51Then paste that function over the existing map function.
00:58The ultimate goal here is to get one file name per document.
01:02This function is a little bit complicated, so let's step through it.
01:05First, we are detecting whether or not the document has an attachments field.
01:11If it doesn't have an attachments field, we just skip over it.
01:14First, we're going to get one file name for each document, and then emit the name field
01:19for the document as the key and the file name as the value.
01:23To actually get that file name, we have a very complicated looking loop.
01:27We are using JavaScript's for-in construct to get the object properties of the attachments field.
01:35This gives us each of the property names as a key.
01:39And then we run two tests on this key.
01:41First, we want to make sure that this key actually belongs to this attachments object,
01:47and then we want to make sure this key does not actually point to a function.
01:51In JavaScript, we are not guaranteed the ordering of the object properties.
01:56So we use these checks to make sure that we just get the file name.
02:00Once we have that file name, we stop the loop and emit.
02:05Now let's click the Run button.
02:10This view has everything we need.
02:12We have the document name as the key, and the file name as the value.
02:17We also have the document ID handy.
02:19This will help us construct the URLs we need to point to the images.
02:23Now, let's click Save As so we can save this view.
02:27For the design document, type menu, and for the view name type items.
02:32So now we have this as a permanent view in our system.
02:37We can also view the raw data.
02:39Remove everything from the question mark down to the root.
02:43So now we have that same data available in JSON format.
02:48Creating a view is helpful for getting a clean list of attachment file names.
02:52With this view in place, you can now easily build applications based around it.
Collapse this transcript
Deploying applications
00:00With photos and names already in the database, the only thing we need now is some way of
00:05displaying the data in a browser.
00:07This is made possible in CouchDB through the concept of applications.
00:11With an application, you can upload HTML and JavaScript that can be served directly to a browser.
00:17First, let's navigate to the restaurant database.
00:21If it's not already appearing, select the Items view.
00:26This Items view has the name of the menu items along with the document IDs and the file name
00:32of the image of the menu item.
00:34Now let's take a look at index.html in the exercise files.
00:40This HTML file runs a jQuery-based application.
00:44When the document is ready, we first get the data from the view.
00:48We do this using the getJSON() method.
00:51Once the data has been retrieved, we then cycle over all the results.
00:55For each result, we construct an image URL using the row.id, and the value.
01:01And then finally, we add the menu item name as well as the image to the photos div.
01:07Let's upload index.html to our database so that we can serve it back to the web browser.
01:13Select Design Documents from the View dropdown, and then select Design Menu.
01:19This is the design document that was created when we created the Items view.
01:23Now select Upload Attachment, and then browse for index.html.
01:29Finally, click Upload.
01:32Now index.html is displayed as an attachment to this design document.
01:37Click on index.html.
01:39All of the menu items now appear along with their photos.
01:43This view automatically updates whenever we change the items.
01:47Let's remove one of these documents so that it updates the menu.
01:52Go back to the homepage for Futon.
01:54Then click restaurant.
01:56And now, let's remove one of these documents.
02:00This is the keylimepie document.
02:02Click Delete Document, and then click Delete.
02:05Now, let's go back to the Photos page and click Refresh.
02:12Now that we've removed key lime pie from the database, our application also updates.
02:17The design document gives us a place to upload HTML, JavaScript, and other assets served to web browsers.
02:23Since the data is already available in JSON format over HTTP, CouchDB has everything we
02:29need to serve, store, and manage a complete web application.
02:33
Collapse this transcript
Securing CouchDB
00:00When a fresh copy of CouchDB is started, there is no authentication in place; anyone can
00:05access any part of it through HTTP.
00:08When you're in development, this is very handy.
00:10However, this is almost never acceptable for production use.
00:14Fortunately, CouchDB allows you to lock down individual databases as well the whole server.
00:19First, let's create an admin.
00:21In the lower right-hand part of the screen you'll notice it says, Welcome to Admin Party!
00:25Everyone is admin. Fix this.
00:28Let's do what it suggests and click Fix this, type in admin as the Username, and then admin as the Password.
00:37We've created an admin but we would still like to lock down the database. Anonymous
00:41users can still read from and write to the database.
00:44Click the Configuration link.
00:45There is a property called require_valid_ user, and it's currently set to false.
00:50Let's double-click and set that to true.
00:53Once you click the check, CouchDB requires a valid user for every operation.
00:58Let's test how this works.
01:01I'm going to go to the Command line to use curl.
01:04Go to Utilities > Terminal.
01:08Let's try to get information about the restaurant database.
01:12Since we haven't logged in, it tells us that authentication is required.
01:17If we supply the username and password, it will now let us in.
01:22Now you don't have to use an admin as the only way of securing your database.
01:26You may have situations where you want certain users to be able to access and modify certain databases.
01:33CouchDB allows this with different roles, but for now we're just going to secure the
01:37whole entire database.
01:39While CouchDB is completely open by default, it is easily secured.
01:43After creating an admin user set require_valid_user to true.
Collapse this transcript
Using Node
00:00While CouchDB is designed so you can store and serve your entire web application from
00:05it, you maybe more comfortable with using a different programming environment for your development.
00:10Many languages and frameworks have libraries ready to connect to CouchDB.
00:15Let's take a look at how you can use Node.js to connect to CouchDB.
00:20You can download Node.js from nodejs.org.
00:23For more information on installing and using Node.js refer to other lynda.com online training library titles.
00:31Let's go to the exercise files.
00:33There is an application folder inside the exercise files.
00:36Let's copy this folder to the Desktop.
00:41Now let's take a look at the contents of this folder.
00:45If you open package.json you'll notice cradle is listed as a dependency.
00:52Cradle is a module that you can use to connect to the CouchDB.
00:55Let's install this dependency now.
00:59I'm going to Applications > Utilities > Terminal, and then I'm changing directory to the application folder.
01:08Once you're inside the application folder, type npm install.
01:16Now cradle and all of its dependencies are installed.
01:21Now let's take a look at db.js.
01:25Db.js requires the cradle module, then it uses the cradle module to create a connection
01:32to the CouchDB instance at 127.0.0.1.
01:37We also supply the port number as well as the username and password.
01:42Finally we tell cradle to use restaurant as the database.
01:46Then we set the db object as our export.
01:51Now let's take a look at list_items.js.
01:57This file first includes db.js by using ./db.
02:03With that object we can then query the view.
02:06This function will query the menu/items view and then log every result to the console.
02:12Let's run list_items.js now.
02:14Go back to Terminal and type node list_items.js.
02:23Cradle retrieved all the information from this view and then we logged every result to the console.
02:30Let's take a look at get_delivery.js.
02:35Here we have the hash for the delivery page, and then inside the callback function we're
02:41logging the result to the console.
02:43Let's run this script now, type in node get_delivery.js.
02:50Finally, let's take a look at doing a save. Open save.js.
02:57Again, we're requiring the DB module, then we're calling the save function.
03:03As the first argument, we're passing an object representing the document that we want to
03:07store in the database.
03:10Notice that we're not supplying an ID. The Cradle module will automatically let couch
03:15supply an ID for us, and then it will return that ID as a part of the response.
03:21Let's run this script now.
03:24Type in node and save.js. So the response came back saying that the save was successful
03:31and then it also gave us the first revision hash, as well as the ID for the document.
03:37Let's look up this document in the CouchDB Futon interface.
03:40I'm in Futon and I'm navigating to restaurant.
03:45You might have to navigate to the next page.
03:50Notice that this document has the name and category we specified in the Node.js script.
03:55The Cradle module for Node.js has everything you need to start working with a CouchDB database.
03:59The Cradle Module makes all of the HTTP requests for you so you don't have to write them manually.
Collapse this transcript
5. NoSQL Trade-Offs
Understanding partitioning
00:00As a database grows in size, maintaining acceptable performance becomes more complex.
00:07Partitioning is one method you use to help manage large and busy databases.
00:11So what is database partitioning?
00:14Database partitioning is splitting data across multiple database servers.
00:19Partitioning is always done by consistent method so you always know where the data is.
00:24There are several different methods that you can use.
00:26One partitioning method you can use is a range.
00:29For instance, if you had your database sorted alphabetically you could have records A through
00:33L on one partition, M through Q on another, and R through Z on a third.
00:39Another method for partitioning is a list.
00:42If you were building a database of books you could categorize them.
00:45You can have textbooks on one partition, cookbooks on another and sci-fi books on a third.
00:51And finally, you could use a hash as a partition method.
00:54With a hash you pass a value into a function, and it returns a value to determine the membership.
01:00So why would you partition in the first place?
01:02One reason might be storage limitations.
01:05If you have a very large dataset, that entire set might not fit on one database server.
01:12Another common reason is for performance.
01:14When you split the load between partitions, it can make it faster to write to the database.
01:19And then finally, you might want to partition for availability.
01:23Placing your data on separate partitions insures that you're always going to be able to get
01:27to the data even if one of the partitions gets busy.
01:30Although there are scenarios where you would want to partition your database, there are
01:34also scenarios where you would not.
01:36When your dataset is small, there's no reason to partition.
01:40It will only increase the complexity of your database server.
01:43Partitioning in relational databases and NoSQL databases is similar.
01:48Relational databases can be partitioned horizontally or vertically.
01:53Horizontal partitioning, which is also known as sharding, puts different rows on different partitions.
01:59Vertical partitioning on the other hand, puts different columns on different partitions.
02:04Partitioning non-relational databases depends on the database type.
02:08Key value and document databases are typically partitioned horizontally.
02:12On the other hand, tabular databases can be horizontally and vertically partitioned.
02:17Many NoSQL databases are designed with partitioning in mind, however smaller databases may not
02:23need partitioning.
02:26Understanding how partitioning works is key to making informed decisions about database performance.
02:32
Collapse this transcript
Understanding the CAP theorem
00:00Finding the ideal database for your application is largely a choice between trade-offs.
00:06The CAP theorem is one concept that can help you understand the trade-offs between different databases.
00:11The CAP theorem was originally proposed by Eric Brewer in 2000.
00:16It was originally conceptualized around network shared data and is often used to generalize
00:20the tradeoffs between different databases.
00:23The CAP theorem centers around three desirable properties; consistency is where all users
00:28get the same data, no matter where they read the data from,
00:32availability ensures users can always read from and write to the database, and finally
00:37partition tolerance ensures that the database works when divided across network.
00:42The theorem states that at most you can only guarantee two of the three properties simultaneously.
00:47So you can have an available partition- tolerant database, a consistent partition-tolerant
00:53database or a consistent available database.
00:57One thing to note is that not all of these properties are necessarily exclusive of each other.
01:03You can have a consistent partition-tolerant database that still has an emphasis on availability,
01:09but you're going to sacrifice either part of your consistency or your partition tolerance.
01:14Relational databases trend towards consistency and availability.
01:19Partition tolerance is something that relational databases typically don't handle very well.
01:24Often you have to write custom code to handle the partitioning of relational databases.
01:29NoSQL databases on the other hand trend towards partition-tolerance.
01:33They are designed with the idea in mind that you're going to be adding more nodes to your
01:37database as it grows.
01:39CouchDB, which we looked at earlier in the course, is an available partition-tolerant database.
01:45That means the data is always available to read from and write to, and that you're able
01:50to add partitions as your database grows.
01:53In some instances, the CAP theorem may not apply to your application.
01:57Depending on the size of your application, CAP tradeoffs may be irrelevant.
02:02If you have a small or a low traffic website, partitions may be useless to you, and in some
02:07cases consistency tradeoffs may not be noticeable.
02:11For instance, the votes on a comment may not show up right away for all users.
02:15This is fine as long as all votes are displayed eventually.
02:19The CAP theorem can be used as a guide for categorizing the tradeoffs between different databases.
02:24Consistency, availability, and partition tolerance are all desirable properties in a database.
02:31While you may not be able to get all three in any single database system, you can use
02:35the CAP theorem to help you decide what to prioritize.
Collapse this transcript
6. Other NoSQL Databases
MongoDB
00:00MongoDB is a document database that uses JavaScript.
00:04This makes it similar to CouchDB, yet there are many features that distinguish it.
00:09First, querying for MongoDB is not done over HTTP.
00:14There are native drivers that are written for each language that you want Mongo.
00:17So that way you connect directly to the database rather than incurring the overhead of HTTP connections.
00:24Mongo does not have permanent CouchDB-style views that you set up.
00:29And finally, MongoDB only has master/slave replication. You can only write to the master
00:34and then that chooses which slave to write to.
00:37Mongo is a consistent partition-tolerant database.
00:41All users should always get the same data back from MongoDB regardless of when they read it.
00:47Documents are partitioned using sharding.
00:49Each partition will have a subset of the documents.
00:53Shards are created based on the key you choose.
00:56This allows you to customize how Mongo partitions the database.
00:59MongoDB is structured around databases, collections, and records.
01:04A single database is intended for a single application. Collections are then used to
01:10organize records that are similar to each other.
01:13Each individual record can have a different structure but you still use collections to organize them.
01:19The JavaScript-based querying that Mongo offers is somewhat similar to SQL.
01:24However, it's still is NoSQL database because you're ultimately using JavaScript to do the querying.
01:31Even though the querying is somewhat similar to SQL, MongoDB still has a schema-free structure.
01:37Individual records can have different field sets.
01:40And finally, MongoDB allows you to define MapReduce functions just like CouchDB does.
01:47While MongoDB is a document database with some things in common with CouchDB, it ultimately
01:52differs in the way you connect, the way it partitions data, and the way you run queries.
Collapse this transcript
Cassandra
00:00Cassandra is an available partition-tolerant database originally developed by Facebook.
00:06While it is similar to CouchDB in relation to the CAP theorem, it is quite different in others.
00:11The querying for Cassandra is not done over HTTP, each language has a native driver that
00:16you use to connect directly to Cassandra.
00:20Storage-wise Cassandra is a cross between a key/value store and a tabular database.
00:25Cassandra is an available partition-tolerant database.
00:29You should always be able to read from and write to Cassandra.
00:33Hardware nodes can be added to Cassandra with no added downtime.
00:37As your application grows, you can add more hardware and Cassandra will keep running.
00:42The consistency of the data in Cassandra can be adjusted but it comes at the expense of
00:47the availability of the data.
00:49In Cassandra, each key maps to one or more columns, and those columns could be grouped
00:55into column families.
00:57Cassandra is queried using its own query language, which is similar to SQL.
01:03The query language is specifically designed for column groups as well as the adjustable
01:07consistency of Cassandra.
01:09Cassandra is very different when compared to CouchDB.
01:12It is a cross between a key/value store and a tabular database with adjustable consistency.
01:18The structure is oriented around columns in column groups.
01:21However, like CouchDB, it is an available partition- tolerant database where new hardware nodes can be added easily.
01:30
Collapse this transcript
Riak
00:00Riak is a document database that is very similar to CouchDB.
00:05Connections to Riak are done over HTTP.
00:08Both Riak and Couch are written in Erlang, a fault-tolerant language where code can be
00:12changed without stopping the system.
00:15However, there are a few differences between Riak and Couch.
00:20You can write MapReduce functions but you can write them in Erlang as well as JavaScript in Riak.
00:25Also, Riak is designed primarily to work on Mac, Linux and other similar systems.
00:31You cannot run Riak easily on Windows.
00:35Riak is an available partition-tolerant database.
00:38You should always be able to read from and write to Riak, and hardware nodes could be added easily.
00:44The structure of Riak is along the lines of bucket/key/value.
00:49Buckets organize documents and you can use the key of a document to retrieve it.
00:53The query syntax for Riak is the same as the Lucene full-text search engine.
00:58Like many other NoSQL databases, you can define MapReduce function pairs.
01:04And finally, you can use key filters with Riak.
01:08Key filters allow you to pick up records with keys matching a certain criteria.
01:12This way you can search for specific documents without having to analyze the contents of the document.
01:18Riak is very similar to CouchDB but there are a few things that set it apart.
01:22Riak allows you to use Lucene syntax and also allows you to filter keys before searching
01:27the document contents.
Collapse this transcript
Redis
00:00Redis and CouchDB are very different databases.
00:03While Redis and CouchDB do not use SQL for querying, the similarities end there.
00:10Querying for Redis is not done over HTTP. You must use native drivers that are written
00:14for each language you want to use.
00:17Redis is a key/value store, unlike CouchDB which allows you to find a document and query
00:22against the contents of that document.
00:25It is designed primarily to work on Mac and Linux.
00:28It does not have Windows support.
00:31And then finally, Redis uses master/slave replication.
00:35Redis is a consistent partition-tolerant database.
00:38Each user should always get the same data back from Redis.
00:42It's possible to write directly to a slave in Redis but this violates the consistency
00:46principle of Redis.
00:48Rather than synchronizing data the way the CouchDB does, data is replicated to multiple slaves in Redis.
00:55Queries in Redis are done primarily by key.
00:58Although Redis is a key/value store, if you have your data stored as a hash in Redis,
01:04it is possible to get individual hash values.
01:07The values stored in Redis do not have to be strings, unlike many other key/value stores.
01:13Redis provides you with lists, hashes and sets as ways of storing data within the value.
01:19Lists are lists of strings and hashes are further key/value pairs that can be nested.
01:25Sets on the other hand are a series of non-repeating values.
01:29Redis is a key/value store that has little in common with CouchDB, the partitioning access
01:35and structures are all different.
01:38Although you can get specific values from hashes in Redis, querying is done primarily
01:42by supplying keys.
01:44
Collapse this transcript
Conclusion
Next steps
00:00Once you have a grasp on how various NoSQL databases work and how they differ from SQL-based
00:06ones, you'll want to build up other skills so you can use them effectively.
00:10Many NoSQL databases use JavaScript.
00:14The lynda.com online training library has several JavaScript related titles.
00:19Also, there are various frameworks and tools that are based on NoSQL databases.
00:25This CouchDB wiki page lists several getting started guides for various languages.
00:31NoSQL databases are being released and expanded.
00:34By learning JavaScript and experimenting with the different software development kits, you'll
00:38be able to pick up new NoSQL databases quickly and use them for your software projects.
00:44
Collapse this transcript


Suggested courses to watch next:

JavaScript Essential Training (5h 31m)
Simon Allardice

SQL Essential Training (2h 22m)
Bill Weinman


Ruby Essential Training (6h 54m)
Kevin Skoglund


Are you sure you want to delete this bookmark?

cancel

Bookmark this Tutorial

Name

Description

{0} characters left

Tags

Separate tags with a space. Use quotes around multi-word tags. Suggested Tags:
loading
cancel

bookmark this course

{0} characters left Separate tags with a space. Use quotes around multi-word tags. Suggested Tags:
loading

Error:

go to playlists »

Create new playlist

name:
description:
save cancel

You must be a lynda.com member to watch this video.

Every course in the lynda.com library contains free videos that let you assess the quality of our tutorials before you subscribe—just click on the blue links to watch them. Become a member to access all 104,141 instructional videos.

get started learn more

If you are already an active lynda.com member, please log in to access the lynda.com library.

Get access to all lynda.com videos

You are currently signed into your admin account, which doesn't let you view lynda.com videos. For full access to the lynda.com library, log in through iplogin.lynda.com, or sign in through your organization's portal. You may also request a user account by calling 1 1 (888) 335-9632 or emailing us at cs@lynda.com.

Get access to all lynda.com videos

You are currently signed into your admin account, which doesn't let you view lynda.com videos. For full access to the lynda.com library, log in through iplogin.lynda.com, or sign in through your organization's portal. You may also request a user account by calling 1 1 (888) 335-9632 or emailing us at cs@lynda.com.

Access to lynda.com videos

Your organization has a limited access membership to the lynda.com library that allows access to only a specific, limited selection of courses.

You don't have access to this video.

You're logged in as an account administrator, but your membership is not active.

Contact a Training Solutions Advisor at 1 (888) 335-9632.

How to access this video.

If this course is one of your five classes, then your class currently isn't in session.

If you want to watch this video and it is not part of your class, upgrade your membership for unlimited access to the full library of 2,025 courses anytime, anywhere.

learn more upgrade

You can always watch the free content included in every course.

Questions? Call Customer Service at 1 1 (888) 335-9632 or email cs@lynda.com.

You don't have access to this video.

You're logged in as an account administrator, but your membership is no longer active. You can still access reports and account information.

To reactivate your account, contact a Training Solutions Advisor at 1 1 (888) 335-9632.

Need help accessing this video?

You can't access this video from your master administrator account.

Call Customer Service at 1 1 (888) 335-9632 or email cs@lynda.com for help accessing this video.

preview image of new course page

Try our new course pages

Explore our redesigned course pages, and tell us about your experience.

If you want to switch back to the old view, change your site preferences from the my account menu.

Try the new pages No, thanks

site feedback

Thanks for signing up.

We’ll send you a confirmation email shortly.


By signing up, you’ll receive about four emails per month, including

We’ll only use your email address to send you these mailings.

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

By signing up, you’ll receive about four emails per month, including

We’ll only use your email address to send you these mailings.

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked