Using external data - D3.js Tutorial

- [Instructor] In a real project you would want to load your data from an external file or database and D3 has some excellent functions to help you with just that. So D3 offers a variety of methods to deal with data import and the nice thing is that it will take care of parsing the data automatically for you. So essentially these functions are AJAX calls, you can get data from many formats including the popular JSON format which you'll be using in this video and once you import the information it will be easily accessible as a Java Script object. Other formats that are pretty common are CSV which you can import from Excel and also TSV which are tab separated values and those are easy to copy and paste from an Excel table or other tables. Once you import the data you can use that data in a call back to create your chart. I've already got a pretty complicated data file in this data folder right here. It's called forcast.json, it's a file that I've gotten from a website called openweathermap.org. Now JSON data can be pretty complicated to understand, especially when it's in a text file like this. So I like to use another website called the JSON Editor Online. And I'm going to copy all this data and switch over to that site. And what you can do here is grab some data. So I've already copied it. I'm just going to paste it right here and then hit this right arrow. And if it's a properly formatted JSON file it's going to sort of restructure the file and it's going to give you the ability to see the structure of the file. So in this object we have five elements. Here are the different elements. And if you open this right here it's just a list of the individual items for the weather data. So if we open one of these you can see that we have some information here called data or dt_txt, that's the date of the entry. And then in here we have weather information as well as some other data. The one that you want is called main, it has information about the temperature at that particular time as well as pressure, humidity, et cetera, et cetera. So we want to get to this temperature data and then eventually I'm also going to grab this date right here. So to get that we just have to remember that we go to the object and then we find the list item and then we have sort of an array of elements and there's 40 of them. These will be our different bars and then once you get to each element then you can go to either dt_txt for the date and then main and if we want say, the temperature we can just go to temp to get that. So let's go back into our file, you don't really need to look at this forecast anymore. And what we're going to do is go to the d3.json method and then read the data file. So it's in the Java Script folder, data and it's called forecast.json. Then we're going to use a call back with that data and then we can do whatever we want. And let's go ahead and put a comment in here. I just want to make a note that if you are working with this data, it's not going to work on your local directory. So if you just take the index.html file and you preview it in your browser it's not going to work, it needs to be uploaded to a server or running on a server. If you're using the exercise files you should have a server running under local host 3000, so that's not going to be an issue. And what I'm going to do next is grab everything, just cut everything out and put it inside this method. This is actually cool because putting it in here is going to protect all this information in here from anything else that I'm doing with this index.html. And it's sort of like doing what we call an iife, or an immediately executed function. And once I have that in here then I can do whatever I want, the data is going to come in this D sort of attribute. And so I'll need to modify my file a little bit so that I can take advantage of the data. So I'm not going to go through a loop like I've done here, or actually I am going to go through a loop, but it's not going to fill this variable called bardata like this. So our loop, it's actually going to be similar to this. And what we're going to do in that loop is go from zero to instead of I equals a hard number, we're going to go to the D list length. And then we're going to push this information right and instead of it being this random number that we got here we can go to an item in the data, so we'll say D list. And then we will use this index to get our main temp. So that's sort of how I was saying you're going to go to list, main and get this temp right here. So that's going to give us all of our temperatures pushed into this bardata array. And I'm going to go ahead and rename this bardata because it's not really bardata anymore, it's really temperatures. And so we'll rename this temperatures. If you're wonderin' how I did that, this particular editor which is Visual Studio Code allows me to select an element and then repeat that selection, most of the other editors have that same functionality, it's usually control, D or command, D depending on which computer you're using. So I'm on a Mac so it's command, D and it just selects a bunch of the same sort of selections over and over. And most of the time you can find it right here. So you can see it's called add next occurrence in this particular editor. Alright, so once I have that data then I can start working with it and what I want to do also is sort of clean up my variables. Right now I sort of create variables whenever I want and it's getting a little bit messy and it's probably not how you should work. It's usually the way that you do things in D3, but it's probably not the best way of doing things. So what you want to usually do is create your variables at the top of your file or your function and then what we're going to do then is create this variable called temperatures that used to be called bardata and this will be an empty array. And then I'm going to move all these other variables. So all this height, width, banwidth, tempcolor, all that stuff is going to just be declared up here as well. So let's just go ahead and use one var and we can indent these to make them look a little nicer. And I'm also going to create another variable section for variables that I'm not just declaring. So variables that are empty. So tempcolor will just sort of go there by itself for right now. And I'm also going to grab, any time I'm using the word var, the keyword var right here I'm going to go ahead and move those variables into just declarations. So I'll take Y scale and put that here. And I'm going to go ahead and do that for all the other var declarations. Alright, I'm also going to clean up the spacing just a little bit, it's again, getting a little bit messy. And I want to sort of keep things consistent so let's indent this. And we'll clean up some of this other spacing right here. Alright, so I finished cleaning things up and usually what I like to do is indent things as consistently as I can. And with D3 you want to indent in whenever you do a dot method and just usually one level. Sometimes when you do enters I like to indent maybe another level and then all theses attributes belong to that enter. So they go on that same level. On mouse over, on mouse out, actually this one should be maybe indented in a little bit. And then all the variables should be at the same levelage. It just makes things a little bit easier to work with in the future. Now let me go ahead and save this. You can see that the bars are showing up really well, everything looks good. So as you can see, D3 is making reading our data extremely simple, it's even taken the JSON file that would normally be super hard to parse and given it to us in a Java Script object. It would do the same thing with either a CSV or a TSV, so whatever format your data is already in, D3 probably has a way of accessing the data using one of it's own AJAX based methods.