Start learning with our library of video tutorials taught by experts. Get started

Drupal 7 Custom Module Development

Importing content using feeds


From:

Drupal 7 Custom Module Development

with Jon Peck

Video: Importing content using feeds

One of the problem domains I encounter often is the need to import content from a data source into Drupal. The United States Department of Energy created the Open Energy Information initiative or OpenEI as a platform to connect the world's energy data. Licensed under a Creative Commons Public Domain one of the many collections of open data they publish is a database of wind farms in the United States. As this module facilitates the creation of a database of wind farms, I'm going to use the freely available CSV to import data.

Watch this entire course now—plus get access to every course in the library. Each course includes high-quality videos taught by expert instructors.

Become a member
Please wait...
Drupal 7 Custom Module Development
2h 57m Advanced Oct 16, 2012

Viewers: in countries Watching now:

Extend your Drupal 7 sites with custom modules, which allow you to create everything from admin interfaces to forms. Author Jon Peck describes how modules extend your base Drupal installation, then walks through how to write your own module with a practical example featuring geo-positioned alternative energy centers. The course also describes how to control access to site features, create new content types, build forms, understand data persistence, embrace coding standards, and much more.

Topics include:
  • Creating your first module
  • Interacting with hooks
  • Working with permissions and roles
  • Controlling access
  • Adding a menu item to an admin interface
  • Using the Form API (FAPI) to quickly create a form
  • Creating custom form validation
  • Manually creating a custom content type
  • Validating user input
  • Importing content using feeds
  • Creating a block
  • Understanding best practices and coding standards
Subjects:
Developer Web CMS Blogs Web Design Programming Languages Web Development
Software:
Drupal
Author:
Jon Peck

Importing content using feeds

One of the problem domains I encounter often is the need to import content from a data source into Drupal. The United States Department of Energy created the Open Energy Information initiative or OpenEI as a platform to connect the world's energy data. Licensed under a Creative Commons Public Domain one of the many collections of open data they publish is a database of wind farms in the United States. As this module facilitates the creation of a database of wind farms, I'm going to use the freely available CSV to import data.

I've trimmed the data set a bit to include only fields that are supported in this content type. One of the many advantages of Drupal is the open platform for developing and sharing content, and functionality, and importing data is one of those areas. The Feeds Module imports and aggregates data as nodes, users, taxonomy terms or even simple database records. Out of the box it supports RSS feeds, CSV files, and much more, and these imports can be performed as one-offs or unscheduled regular intervals.

This particular demonstration is not going to require any custom development for the importer which may seem a little counterintuitive given the name of the course. The point is that while I can do something manually, when somebody else has built and maintained a great wheel that tens of thousands of other people use on a daily basis, there's no reason to reinvent it. Feeds does have some requirements. Download and extract the following modules if they're not already installed. Feeds which is the base module, ctools which is required by feeds; it's also a great development library, and job_scheduler which is required by feeds, and facilitates the scheduling of imports.

When they are available go to the Module list, and enable Feeds Admin UI. The required modules will be enabled as needed. Now that feeds is enabled, I need to setup the mechanism for importing data. Go to Structure then Feeds importers. Each Feed importer can be used to define how data is handled upon import, including what factor will be used, mapping, standards, and so forth.

Click Add Importer, for the name I'll use OpenEI Wind Farms, for the description CSV Import of OpenEI Wind Farm Database, click Create. The first thing that needs to be configured is the basic settings which defines general behavior of the importer, click Settings, disable Periodic Import by selecting Off then click Save.

Next, configure the Fetcher which gets the data from a source. There are two options right now; File upload and HTTP Fetcher, switch to a File upload and click Save. Then change the Parser. By default Feeds will parse RSS and Atom Feeds. Switch to the CSV parser, then click Save. By default, the Processor used will be the Node processor which creates and updates nodes from the parsed content.

This is what I want, so no change is needed. Go to the Node Processor Settings, under Content type select Wind Farm, then Save. Finally, define the Mapping. This will define the mapping of the CSV headers to the content type fields. On the left, the Source is the CSV header name, and on the right the Target is the content type field. So we'll add them one at a time. So the first will be Facility Name, I will map it to Title which is the default name of the node title.

In the content type I named this Facility Name, but this interface doesn't check that deeply for, click Add. Next, Facility; I will map this to the Description, just so there's some content. NumberOfUnits, which I will map to Number of Units. Latitude, maps to Latitude.

Longitude, maps to Longitude. And finally, WindTurbineManufacturer. This maps to the Turbine Manufacturer, click Save. That was fairly laborious, wasn't it? Fortunately, this is a one time thing. When I said that custom development wasn't needed I was telling the truth. I'm not writing an importer, however, code will be involved.

One of the tabs on this page is Export. This generates PHP code that can be embedded in a module that will programmatically create the feed importer when that module is enabled. Click on Export now, Select All and Copy. Go to the IDE and open windfarms.module, scroll to the end, then create a doc block. Modules can make hooks available to other modules and feeds is no exception. To provide a default feeds importer, I need to implement hook, feeds, importer, default.

Implements hook_feeds_importer_default(); function windfarms_feeds_importer_default(), which takes no parameters. Feeds exports into a variable called Feeds Importer. Hook_feeds_importer_default returns an array of these feeds. Start with a variable called export with an empty array. $export = array().

Paste the contents of the export in. Then indent it, so it lines up cleanly. Remove the long comment about disabling. At the end, add the feeds importer to export with the key that matches the machine name of the exported feed, which can be found in the feeds importer ID property. In this case, openei_wind_farms; $export 'openei_wind_farms' = $feeds_importer.

Then return $export. Before this will work, Feeds needs to know that this default exists. Ctools has a plug-in API that feeds leverages. So the Wind Farms module needs to let ctools know that it's providing defaults. Implements hook_ctools_plugin_api(); function windfarms_ctools_plugin_api().

The API takes two parameters: the module and the API both strings and both sent to the empty string. ($module = '', $api = ''). Add some logic. If the feeds module and the API question is the Feeds importer default, so if ($module == 'feeds' && $api == 'feeds_importer_ default') then you tell ctools that the API version is 1; return array ('version' => 1). Save then return to the browser.

Go to Structure then Feeds importers, the status is now overridden. If it's not, you'll need to clear the cache, click Revert to go back to the version stored in code. The importer now is using the configuration stored in the module as indicated by the status default. Now that the windfarms module depends on feeds, that information needs to be made available to Drupal, so we can require feeds for enabling Wind Farms. Go back to the IDE, and open the windfarms.info file.

While feeds requires several other modules to function, Wind Farms explicitly depends on feeds, and feeds can handle its own dependencies. Drupal is intelligent enough to enable everything required if you just enable windfarms. Add a line to the end of the info file. Dependencies can contain multiple values so using this standard INI format for arrays, set the key as dependencies and the value to fields, Save. All the groundwork is complete.

The windfarms module will now import, and the configuration is now stored in code. Go back to the browser, then click on the link to the Import page. There should be an option OpenEI Wind Farms, click it now. Go to File, and Choose File. Browse to the location of the windfarms. csv file, then click Open and Import.

The Status Message indicates that 948 nodes are created. Return to the homepage. Returning to the homepage, the latest wind farms to be imported are now shown. Click on one of the imported items. As it looks now, it's not very interesting. In the next segment, I'll add a Google Map of the facility which will be much more useful.

Find answers to the most frequently asked questions about Drupal 7 Custom Module Development.


Expand all | Collapse all
Please wait...
Q: gmap3_tools is not working the way I expected it to. What version of gmap3_tools should I be using?
A: Use the free exercise file containing the version of gmap3_tools used for recording; the published version of the module on drupal.org has changed since recording and is not backwards compatible.
Q: I attempted to run the Drupal site root from the project files, but the site isn't loading. Why not?
A: The Drupal configuration file in sites/default/settings.php contains database configuration specific to the environment used to record the movie. This may be different than your environment. Edit the file and search for "windfarms" - you may need to change the database host, username, password, db name and port to match your specific environment.
Share a link to this course
Please wait... Please wait...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.
Upgrade now


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

join now

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Drupal 7 Custom Module Development.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Welcome to the redesigned course page.

We’ve moved some things around, and now you can



Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked