In this video, Kathryn shows you how to use the urllib module to get content from the internet. She then shows you how to use the JSON module to decode this content and get what you need from it.
- [Instructor] We are going to use the urllib module to get content from the internet. To do this, we're going to have to import some things. So we are going to import the urllib.request, and this is just a class that we're going to use. And then we're going to import json and we're also going to import the textwrap module. And I'll explain how these fit in in a minute. But in this video, we are going to use the Google Books API. And so what's going to happen is we are going to give Google Books an ISBN number, and then it's going to take that ISBN number and get some data on that specific book.
And so this is the URL that we are going to use to do this. And so in order to make this get request, we are going to go with urllib.request, so using this urllib module, urlopen, which is the function we're using, and I'm just going to copy and paste this right inside of here, inside of a string. To find out more information about the Google Books API and how this URL works, go look at the information down below.
This URL will also be down there, and so you can just copy and paste it right into your code. So we'll go ahead and close this parenthese. And we'll go as f. So we're saying if this request works, then we're going to go ahead and call it f. With f, we are going to create a variable called text, and we're going to go f.read, which is going to read everything that the request returned. Then we're going to go ahead and decode this text into UTF-8, create a variable called decoded text, we'll call it text.decode.
And we'll have utf-8. The reason we're doing this is so that we can have it decoded so we can print it out to the user. And so we'll go print, using the textwrap module that we learned about in a different video, .fill, we'll have our decoded text, and we'll go ahead and go with =50. Printing this out, we see this entire JSON object is what's returned by this API. And we can actually go to this link and that's exactly what we'll see.
So if we go ahead and copy this and put it into our browser, paste it here, this is exactly what was printed out to us in the console in our Python. And so now we have access to this JSON object. Inside of our Python code, we can iterate through this JSON object and get the information we want. Maybe the the title, some authors, et cetera. All the information about this specific book with this given API is there. If we wanted to see information about a different book, all we'd have to do is change the ISBN number here.
So let's go ahead and load the JSON, so that way we can manipulate the data and print specific things out to the user. So we'll add a little print statement here to separate it from the stuff that was returned from the request, and then we'll go obj = json.loads, so using the JSON module, (decoded text). So what the JSON module is doing is it's loading this decoded text, so that way we can iterate it inside of our python code. With this object in the right format, we could go print(obj['kind']).
And so if you notice here, one of the attributes of this object is kind. So when we print this out, we should get books#volumes in return. Printing this out, that's exactly what we get. And although it may be hard to read here, you can always look at this document here by just going to the URL link and you could see like, oh, kind is one attribute, and you could go through all of these selecting the right attributes and get the information you want. Trying this in another example, we're going to go print(obj['items']  ['searchInfo'] ['textSnippet']).
And we're actually going to go back to our browser to kind of see what exactly this is accessing. And so in our code, we were accessing the object. We were accessing the items. We were picking the first item that we have here, which is this entire thing. Scrolling down, we'll see oh, one of the things inside of our items for the first item is search info, so we select that. Inside of search info, we have text snippet, and for the value of that, we have this description that kind of describes the book that we're talking about with this ISBN number.
So that's what we're going to print out in our code. So we had our items, we picked the first item, we accessed the search info, and then inside of that, we accessed textSnippet. And so printing this, we get the little snippet that we saw in our browser. It's important to note that we have zero here, and that's because items is a list. With this here, we see, oh, we are going to list some items and in order to access stuff in a list, they're accessed by index, with the index starting at zero.
So the first item is going to have index zero. For these other things, like kind here, kind is an attribute of this object, because there's a curly brace here. So that's why we just have this accessing of the given attribute here. It's important to note that we have zero here, and that's because we're accessing the first thing inside of the items list. Going back to our browser, we can see this more clearly. We see that items is a list, and it only has one item in this list.
We have this entire huge thing, and to access this, all we do is items at zero, because lists are accessed by index, and they start at zero and go all the way to the list size minus one. And here, since the list size is only one, the only index that we can access is zero. And once we're inside the first item of the list, then we're looking for search info, and that's what we see here and that's what we go into and then we access textSnippet, which is an attribute of the search info, and then we ultimately get this description that we then print out to the user.
It's very useful to know what form your data will be in in order to retrieve what you want. Knowing this module is also super-helpful if you need to get data from an in point, in-A project. Here, it was the Google Books API, but in the future, it could be whatever API you'd like.
- Working with logical and comparison operators
- Getting a list of numbers with the range() and list() functions
- Using mathematical functions such as round(), abs(), and pow()
- Calculating a given input's length
- Importing and using the math module
- Reading a user's command-line arguments
- Getting the current time
- Formatting dates and times with datetime
- Creating a timer
- Using urllib to get content from the Internet
- Using the JSON module to decode content