Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Retrieving the full-year records

From: Code Clinic: PHP

Video: Retrieving the full-year records

The full year records of deep more weather data And save that as get_full_years.php.

Retrieving the full-year records

The full year records of deep more weather data are located in the data/dm directory of the ponderay website. They're all named the same way. Environmental, underscore, data. Underscore followed by the year then .txt. According to this listing, the files are quite large. The ten files add up to ta total of just over 29 megabytes. So rather than attempt to access them remotely, I think it's best to download them and then work with them locally.

One way to get copies of these files is to right click the link of each one and then save link as. But since we're using PHP, let's create a script to do it for us. The first thing that we need is the URL. So if we can copy from the address from the browser, and then go to our editing program. And we need to create a new PHP file. And save that as get_full_years.php.

By default, PHP scripts automatically terminate after 30 seconds. So unless you've got a really fast internet connected, that's too short to download all the files. So we need to remove the time limit, we can do that by changing he PHP directive using ini set. So ini_set, and the argument of that is the directive, which is max_execution_time.

And the second argument is a string which we set to zero. So that will remove the max execution time. Next we need a variable for the URL, and that needs to be a string and we can paste in the URL that we copied from the Pondaray website. And all of the files begin with environmental_data_ ,so we can add that to the end of the URL. And then we create a loop to fetch the files for 2001 through to 2010.

So we'll have a full loop. We'll initialize year at 2001. We'll increment it until it gets less than or equal to 2010. Have been incremented each time the loop runs. Inside the loop, we can use file get contents to get the file from the remote server. And then file put contents to save it to our local disk. And we'll use some conditional statements to make sure that everything has gone okay.

So, first we need to get it. So if, and they will store the file contents as data, file get contents, and the argument to that will be the URL. Which will concatenate with the year, and then we will concatenate .txt as a string. So I then we'll need to put the contents using file put contents. So, if file put contents, and the first argument to that will be where we're going to put the file.

We'll put it in a folder called rawdata. Finish that with a trailing slash. And then, we'll concatenate onto that the current year. And onto that string.txt. And the second argument is the data that we're going to putting in this data file, which is in the data variable. So, if we've got that, we can say that we've saved the year. And if there was a problem, we'll say so.

And then if anything has gone wrong, a message saying problem getting the file. So, this script will go to the Lake Pend Oreille website, it will get the environmental data followed by the .txt, get that, it will save it in the raw data folder locally, and we should have everything locally to work with. I've already created the folder where I'm going to store all this raw data. So, all that's needed is to save that and then to load it into a browser. Now, depending on your connection, it could take a few seconds to a few minutes.

But if all goes well, you should see a list of years that have been saved. And there we are. We've saved all the years, everything seems to have gone fine. So, let's go back to my editing program, and there is raw data. And if we open that folder, there are all the text files in there. Everything has been downloaded successfully. So, the next step is to inspect the content of the files in preparation for filtering and loading the data into the database.

Show transcript

This video is part of

Image for Code Clinic: PHP
Code Clinic: PHP

36 video lessons · 2499 viewers

David Powers
Author

 
Expand all | Collapse all
  1. 8m 49s
    1. Welcome
      1m 18s
    2. What you should know before watching this course
      4m 40s
    3. Using the exercise files
      1m 56s
    4. Getting the most from Code Clinic
      55s
  2. 1h 34m
    1. Introducing Lake Pend Oreille
      5m 4s
    2. Overview of my solution
      3m 22s
    3. Setting up the database
      4m 55s
    4. Retrieving the full-year records
      5m 20s
    5. Processing the full-year records
      10m 49s
    6. Priming the date_recorded column
      7m 47s
    7. Inserting the full-year data into the database
      6m 48s
    8. Processing individual days
      6m 54s
    9. Retrieving a year's data a day at a time
      7m 41s
    10. Keeping the data up to date
      4m 27s
    11. Calculating the mean and median values
      7m 21s
    12. Creating a web service
      6m 21s
    13. Getting the required data
      6m 50s
    14. Building the web service response
      10m 38s
  3. 49m 47s
    1. Identify the image subset
      3m 14s
    2. Overview of my solution
      3m 19s
    3. Setting up the files
      2m 43s
    4. Calculating the scaling ratio
      6m 28s
    5. Using a custom class to scale images
      6m 25s
    6. Generating the thumbnails
      7m 49s
    7. Using subimage-search in ImageMagick
      4m 32s
    8. Analyzing the image statistics
      4m 45s
    9. Finding the cropped images
      7m 13s
    10. Running the script and displaying the results
      3m 19s
  4. 39m 3s
    1. A classic CS interview question NEW
      2m 4s
    2. Overview of my solution NEW
      3m 48s
    3. Finding all possible combinations NEW
      6m 26s
    4. Detecting horizontal attacks programmatically NEW
      4m 8s
    5. Implementing the checkLayout() function NEW
      6m 25s
    6. Rotating the chessboard NEW
      4m 59s
    7. Eliminating duplicate solutions NEW
      6m 34s
    8. Displaying the unique solutions NEW
      4m 39s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Code Clinic: PHP.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Are you sure you want to delete this note?

No

Your file was successfully uploaded.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.