Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member
The full year records of deep more weather data are located in the data/dm directory of the ponderay website. They're all named the same way. Environmental, underscore, data. Underscore followed by the year then .txt. According to this listing, the files are quite large. The ten files add up to ta total of just over 29 megabytes. So rather than attempt to access them remotely, I think it's best to download them and then work with them locally.
One way to get copies of these files is to right click the link of each one and then save link as. But since we're using PHP, let's create a script to do it for us. The first thing that we need is the URL. So if we can copy from the address from the browser, and then go to our editing program. And we need to create a new PHP file. And save that as get_full_years.php.
By default, PHP scripts automatically terminate after 30 seconds. So unless you've got a really fast internet connected, that's too short to download all the files. So we need to remove the time limit, we can do that by changing he PHP directive using ini set. So ini_set, and the argument of that is the directive, which is max_execution_time.
And the second argument is a string which we set to zero. So that will remove the max execution time. Next we need a variable for the URL, and that needs to be a string and we can paste in the URL that we copied from the Pondaray website. And all of the files begin with environmental_data_ ,so we can add that to the end of the URL. And then we create a loop to fetch the files for 2001 through to 2010.
So we'll have a full loop. We'll initialize year at 2001. We'll increment it until it gets less than or equal to 2010. Have been incremented each time the loop runs. Inside the loop, we can use file get contents to get the file from the remote server. And then file put contents to save it to our local disk. And we'll use some conditional statements to make sure that everything has gone okay.
So, first we need to get it. So if, and they will store the file contents as data, file get contents, and the argument to that will be the URL. Which will concatenate with the year, and then we will concatenate .txt as a string. So I then we'll need to put the contents using file put contents. So, if file put contents, and the first argument to that will be where we're going to put the file.
We'll put it in a folder called rawdata. Finish that with a trailing slash. And then, we'll concatenate onto that the current year. And onto that string.txt. And the second argument is the data that we're going to putting in this data file, which is in the data variable. So, if we've got that, we can say that we've saved the year. And if there was a problem, we'll say so.
And then if anything has gone wrong, a message saying problem getting the file. So, this script will go to the Lake Pend Oreille website, it will get the environmental data followed by the .txt, get that, it will save it in the raw data folder locally, and we should have everything locally to work with. I've already created the folder where I'm going to store all this raw data. So, all that's needed is to save that and then to load it into a browser. Now, depending on your connection, it could take a few seconds to a few minutes.
But if all goes well, you should see a list of years that have been saved. And there we are. We've saved all the years, everything seems to have gone fine. So, let's go back to my editing program, and there is raw data. And if we open that folder, there are all the text files in there. Everything has been downloaded successfully. So, the next step is to inspect the content of the files in preparation for filtering and loading the data into the database.
Get unlimited access to all courses for just $25/month.Become a member
82 Video lessons · 101634 Viewers
61 Video lessons · 88389 Viewers
71 Video lessons · 72227 Viewers
56 Video lessons · 103975 Viewers
Access exercise files from a button right under the course name.
Search within course videos and transcripts, and jump right to the results.
Remove icons showing you already watched videos if you want to start over.
Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.
Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.
Your file was successfully uploaded.