Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Importing content using feeds

From: Drupal 7 Custom Module Development

Video: Importing content using feeds

One of the problem domains I encounter often is the need to import content from a data source into Drupal. The United States Department of Energy created the Open Energy Information initiative or OpenEI as a platform to connect the world's energy data. Licensed under a Creative Commons Public Domain one of the many collections of open data they publish is a database of wind farms in the United States. As this module facilitates the creation of a database of wind farms, I'm going to use the freely available CSV to import data.

Importing content using feeds

One of the problem domains I encounter often is the need to import content from a data source into Drupal. The United States Department of Energy created the Open Energy Information initiative or OpenEI as a platform to connect the world's energy data. Licensed under a Creative Commons Public Domain one of the many collections of open data they publish is a database of wind farms in the United States. As this module facilitates the creation of a database of wind farms, I'm going to use the freely available CSV to import data.

I've trimmed the data set a bit to include only fields that are supported in this content type. One of the many advantages of Drupal is the open platform for developing and sharing content, and functionality, and importing data is one of those areas. The Feeds Module imports and aggregates data as nodes, users, taxonomy terms or even simple database records. Out of the box it supports RSS feeds, CSV files, and much more, and these imports can be performed as one-offs or unscheduled regular intervals.

This particular demonstration is not going to require any custom development for the importer which may seem a little counterintuitive given the name of the course. The point is that while I can do something manually, when somebody else has built and maintained a great wheel that tens of thousands of other people use on a daily basis, there's no reason to reinvent it. Feeds does have some requirements. Download and extract the following modules if they're not already installed. Feeds which is the base module, ctools which is required by feeds; it's also a great development library, and job_scheduler which is required by feeds, and facilitates the scheduling of imports.

When they are available go to the Module list, and enable Feeds Admin UI. The required modules will be enabled as needed. Now that feeds is enabled, I need to setup the mechanism for importing data. Go to Structure then Feeds importers. Each Feed importer can be used to define how data is handled upon import, including what factor will be used, mapping, standards, and so forth.

Click Add Importer, for the name I'll use OpenEI Wind Farms, for the description CSV Import of OpenEI Wind Farm Database, click Create. The first thing that needs to be configured is the basic settings which defines general behavior of the importer, click Settings, disable Periodic Import by selecting Off then click Save.

Next, configure the Fetcher which gets the data from a source. There are two options right now; File upload and HTTP Fetcher, switch to a File upload and click Save. Then change the Parser. By default Feeds will parse RSS and Atom Feeds. Switch to the CSV parser, then click Save. By default, the Processor used will be the Node processor which creates and updates nodes from the parsed content.

This is what I want, so no change is needed. Go to the Node Processor Settings, under Content type select Wind Farm, then Save. Finally, define the Mapping. This will define the mapping of the CSV headers to the content type fields. On the left, the Source is the CSV header name, and on the right the Target is the content type field. So we'll add them one at a time. So the first will be Facility Name, I will map it to Title which is the default name of the node title.

In the content type I named this Facility Name, but this interface doesn't check that deeply for, click Add. Next, Facility; I will map this to the Description, just so there's some content. NumberOfUnits, which I will map to Number of Units. Latitude, maps to Latitude.

Longitude, maps to Longitude. And finally, WindTurbineManufacturer. This maps to the Turbine Manufacturer, click Save. That was fairly laborious, wasn't it? Fortunately, this is a one time thing. When I said that custom development wasn't needed I was telling the truth. I'm not writing an importer, however, code will be involved.

One of the tabs on this page is Export. This generates PHP code that can be embedded in a module that will programmatically create the feed importer when that module is enabled. Click on Export now, Select All and Copy. Go to the IDE and open windfarms.module, scroll to the end, then create a doc block. Modules can make hooks available to other modules and feeds is no exception. To provide a default feeds importer, I need to implement hook, feeds, importer, default.

Implements hook_feeds_importer_default(); function windfarms_feeds_importer_default(), which takes no parameters. Feeds exports into a variable called Feeds Importer. Hook_feeds_importer_default returns an array of these feeds. Start with a variable called export with an empty array. $export = array().

Paste the contents of the export in. Then indent it, so it lines up cleanly. Remove the long comment about disabling. At the end, add the feeds importer to export with the key that matches the machine name of the exported feed, which can be found in the feeds importer ID property. In this case, openei_wind_farms; $export 'openei_wind_farms' = $feeds_importer.

Then return $export. Before this will work, Feeds needs to know that this default exists. Ctools has a plug-in API that feeds leverages. So the Wind Farms module needs to let ctools know that it's providing defaults. Implements hook_ctools_plugin_api(); function windfarms_ctools_plugin_api().

The API takes two parameters: the module and the API both strings and both sent to the empty string. ($module = '', $api = ''). Add some logic. If the feeds module and the API question is the Feeds importer default, so if ($module == 'feeds' && $api == 'feeds_importer_ default') then you tell ctools that the API version is 1; return array ('version' => 1). Save then return to the browser.

Go to Structure then Feeds importers, the status is now overridden. If it's not, you'll need to clear the cache, click Revert to go back to the version stored in code. The importer now is using the configuration stored in the module as indicated by the status default. Now that the windfarms module depends on feeds, that information needs to be made available to Drupal, so we can require feeds for enabling Wind Farms. Go back to the IDE, and open the windfarms.info file.

While feeds requires several other modules to function, Wind Farms explicitly depends on feeds, and feeds can handle its own dependencies. Drupal is intelligent enough to enable everything required if you just enable windfarms. Add a line to the end of the info file. Dependencies can contain multiple values so using this standard INI format for arrays, set the key as dependencies and the value to fields, Save. All the groundwork is complete.

The windfarms module will now import, and the configuration is now stored in code. Go back to the browser, then click on the link to the Import page. There should be an option OpenEI Wind Farms, click it now. Go to File, and Choose File. Browse to the location of the windfarms. csv file, then click Open and Import.

The Status Message indicates that 948 nodes are created. Return to the homepage. Returning to the homepage, the latest wind farms to be imported are now shown. Click on one of the imported items. As it looks now, it's not very interesting. In the next segment, I'll add a Google Map of the facility which will be much more useful.

Show transcript

This video is part of

Image for Drupal 7 Custom Module Development
Drupal 7 Custom Module Development

24 video lessons · 5777 viewers

Jon Peck
Author

 

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ .

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Drupal 7 Custom Module Development.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member ?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferences from the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Learn more, save more. Upgrade today!

Get our Annual Premium Membership at our best savings yet.

Upgrade to our Annual Premium Membership today and get even more value from your lynda.com subscription:

“In a way, I feel like you are rooting for me. Like you are really invested in my experience, and want me to get as much out of these courses as possible this is the best place to start on your journey to learning new material.”— Nadine H.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.