Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member
One of the problem domains I encounter often is the need to import content from a data source into Drupal. The United States Department of Energy created the Open Energy Information initiative or OpenEI as a platform to connect the world's energy data. Licensed under a Creative Commons Public Domain one of the many collections of open data they publish is a database of wind farms in the United States. As this module facilitates the creation of a database of wind farms, I'm going to use the freely available CSV to import data.
I've trimmed the data set a bit to include only fields that are supported in this content type. One of the many advantages of Drupal is the open platform for developing and sharing content, and functionality, and importing data is one of those areas. The Feeds Module imports and aggregates data as nodes, users, taxonomy terms or even simple database records. Out of the box it supports RSS feeds, CSV files, and much more, and these imports can be performed as one-offs or unscheduled regular intervals.
This particular demonstration is not going to require any custom development for the importer which may seem a little counterintuitive given the name of the course. The point is that while I can do something manually, when somebody else has built and maintained a great wheel that tens of thousands of other people use on a daily basis, there's no reason to reinvent it. Feeds does have some requirements. Download and extract the following modules if they're not already installed. Feeds which is the base module, ctools which is required by feeds; it's also a great development library, and job_scheduler which is required by feeds, and facilitates the scheduling of imports.
When they are available go to the Module list, and enable Feeds Admin UI. The required modules will be enabled as needed. Now that feeds is enabled, I need to setup the mechanism for importing data. Go to Structure then Feeds importers. Each Feed importer can be used to define how data is handled upon import, including what factor will be used, mapping, standards, and so forth.
Click Add Importer, for the name I'll use OpenEI Wind Farms, for the description CSV Import of OpenEI Wind Farm Database, click Create. The first thing that needs to be configured is the basic settings which defines general behavior of the importer, click Settings, disable Periodic Import by selecting Off then click Save.
Next, configure the Fetcher which gets the data from a source. There are two options right now; File upload and HTTP Fetcher, switch to a File upload and click Save. Then change the Parser. By default Feeds will parse RSS and Atom Feeds. Switch to the CSV parser, then click Save. By default, the Processor used will be the Node processor which creates and updates nodes from the parsed content.
This is what I want, so no change is needed. Go to the Node Processor Settings, under Content type select Wind Farm, then Save. Finally, define the Mapping. This will define the mapping of the CSV headers to the content type fields. On the left, the Source is the CSV header name, and on the right the Target is the content type field. So we'll add them one at a time. So the first will be Facility Name, I will map it to Title which is the default name of the node title.
In the content type I named this Facility Name, but this interface doesn't check that deeply for, click Add. Next, Facility; I will map this to the Description, just so there's some content. NumberOfUnits, which I will map to Number of Units. Latitude, maps to Latitude.
Longitude, maps to Longitude. And finally, WindTurbineManufacturer. This maps to the Turbine Manufacturer, click Save. That was fairly laborious, wasn't it? Fortunately, this is a one time thing. When I said that custom development wasn't needed I was telling the truth. I'm not writing an importer, however, code will be involved.
One of the tabs on this page is Export. This generates PHP code that can be embedded in a module that will programmatically create the feed importer when that module is enabled. Click on Export now, Select All and Copy. Go to the IDE and open windfarms.module, scroll to the end, then create a doc block. Modules can make hooks available to other modules and feeds is no exception. To provide a default feeds importer, I need to implement hook, feeds, importer, default.
Implements hook_feeds_importer_default(); function windfarms_feeds_importer_default(), which takes no parameters. Feeds exports into a variable called Feeds Importer. Hook_feeds_importer_default returns an array of these feeds. Start with a variable called export with an empty array. $export = array().
Paste the contents of the export in. Then indent it, so it lines up cleanly. Remove the long comment about disabling. At the end, add the feeds importer to export with the key that matches the machine name of the exported feed, which can be found in the feeds importer ID property. In this case, openei_wind_farms; $export 'openei_wind_farms' = $feeds_importer.
Then return $export. Before this will work, Feeds needs to know that this default exists. Ctools has a plug-in API that feeds leverages. So the Wind Farms module needs to let ctools know that it's providing defaults. Implements hook_ctools_plugin_api(); function windfarms_ctools_plugin_api().
The API takes two parameters: the module and the API both strings and both sent to the empty string. ($module = '', $api = ''). Add some logic. If the feeds module and the API question is the Feeds importer default, so if ($module == 'feeds' && $api == 'feeds_importer_ default') then you tell ctools that the API version is 1; return array ('version' => 1). Save then return to the browser.
Go to Structure then Feeds importers, the status is now overridden. If it's not, you'll need to clear the cache, click Revert to go back to the version stored in code. The importer now is using the configuration stored in the module as indicated by the status default. Now that the windfarms module depends on feeds, that information needs to be made available to Drupal, so we can require feeds for enabling Wind Farms. Go back to the IDE, and open the windfarms.info file.
While feeds requires several other modules to function, Wind Farms explicitly depends on feeds, and feeds can handle its own dependencies. Drupal is intelligent enough to enable everything required if you just enable windfarms. Add a line to the end of the info file. Dependencies can contain multiple values so using this standard INI format for arrays, set the key as dependencies and the value to fields, Save. All the groundwork is complete.
The windfarms module will now import, and the configuration is now stored in code. Go back to the browser, then click on the link to the Import page. There should be an option OpenEI Wind Farms, click it now. Go to File, and Choose File. Browse to the location of the windfarms. csv file, then click Open and Import.
The Status Message indicates that 948 nodes are created. Return to the homepage. Returning to the homepage, the latest wind farms to be imported are now shown. Click on one of the imported items. As it looks now, it's not very interesting. In the next segment, I'll add a Google Map of the facility which will be much more useful.
Get unlimited access to all courses for just $25/month.Become a member
82 Video lessons · 97410 Viewers
61 Video lessons · 84638 Viewers
71 Video lessons · 68850 Viewers
56 Video lessons · 101308 Viewers
Access exercise files from a button right under the course name.
Search within course videos and transcripts, and jump right to the results.
Remove icons showing you already watched videos if you want to start over.
Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.
Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.
Your file was successfully uploaded.