There are a number of articles on the internet that explain how to write custom modules in magento but for a new developer that has never worked with Zend and has little experience coding for Magento even the simplest task can be daunting. In this article I will go over building a simple twitter module with database and session caching that is configurable via the administrative panel.

Twitters API Call

http://twitter.com/statuses/user_timeline/USERNAME.json?count=NUMBER_OF_TWEETS_TO_RETURN

This is the call to the Twitter API that retrieves the last tweets made by the specified user. Don’t ever try to throw this in one of your view templates making a call to the Twitter API every page you visit as you will only get blocked by Twitter. Read number three on the API Terms of Service. The module that I will develop here caches the result of the API call in the database and the users session. If another user visits the site before the database cache times out, the user will fetch the result from the database instead of making an unnecessary API call. Once a user fetches the result, any subsequent page that they visit will grab the result from the session until it expires. Both timeouts will be set in the admin panel of the module.

Codepools and telling magento about your new module

Magento has three codepools: core, community and local. The core codepool is dedicated for all of magentos system modules, the community codepool is for modules developed by magentos partners and the local codepool is for you as the developer to write custom code. Lets set up a new package and module for our new twitter module. Since I wrote this for my company (Tech And House) my package will be TechAndHouse and the module will be Twitter. (This module is not used on our site, just something I was playing with to document magento module creation). Set up the following folder structure under /app/code/local/:

Next, declare your shell module and its codepool. Create a file named TechAndHouse_Tweet.xml under /app/etc/modules/ with the following:

<?xml version="1.0"?>
<config>
    <modules>
        <TechAndHouse_Tweet>
            <active>true</active>
            <codePool>local</codePool>
        </TechAndHouse_Tweet>
    </modules>
</config>

SQL setup

Now that magento knows about our module, lets set up the database table that will cache the API call. Create config.xml under /app/code/local/TechAndHouse/Tweet/etc/ and define the modules setup handler:

<?xml version="1.0"?>
<config>
    <modules>
        <TechAndHouse_Tweet>
          <version>0.1.0</version>
        </TechAndHouse_Tweet>
    </modules>
 
    <global>
      <resources>
 
        <tweet_setup>
          <setup>
            <module>TechAndHouse_Tweet</module>
          </setup>
          <connection>
            <use>core_setup</use>
          </connection>
        </tweet_setup>
 
        <tweet_write>
          <connection>
            <use>core_write</use>
          </connection>
        </tweet_write>
 
        <tweet_read>
          <connection>
            <use>core_read</use>
          </connection>
        </tweet_read>
 
      </resources>
    </global>
</config>

Create install script, mysql4-install-0.1.0.php to create table th_tweet in the following directory: /app/code/local/TechAndHouse/Tweet/sql/tweet_setup/

<?php
  $installer = $this;
  $installer->startSetup();
 
  $installer->run("
    DROP TABLE IF EXISTS {$this->getTable('th_tweet')};
 
    CREATE TABLE {$this->getTable('th_tweet')} (
      `tweet_id` int(11) NOT NULL AUTO_INCREMENT,
      `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      `twitter_id` bigint(20) NOT NULL,
      `text` text NOT NULL,
      PRIMARY KEY (`tweet_id`)
    ) ENGINE=MyISAM  DEFAULT CHARSET=latin1
  ");
 
  $installer->endSetup();

With this code in place, visit any page on your store and the table will be created. What happens is that upon instantiation, magento reads the /app/etc/modules directory to get information about all its active modules. Then the config.xml file of each module is read. If a SQL setup handler is found then the appropriate version number is read after comparing to the core_resource table. If the core_resource table does not have a SQL handler listed, magento runs the installer. If the version number in the config file is greater than the version number in the core_resource table, magento runs the proper updater SQL file. For example, if the version number in the config file is 0.1.0 and in the config file the version number is 0.2.0, then magento will run the mysql4-upgrade-0.1.0-0.2.0.php.

Administration Section And ACL

Create a new file, system.xml in /app/code/local/TechAndHouse/Tweet/etc/ with the following contents:

<?xml version="1.0"?>
<config>
    <tabs>
        <techandhouse translate="label">
            <label>Tech And House Modules</label>
            <sort_order>100</sort_order>
        </techandhouse>
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            <label>Twitter</label>
            <tab>techandhouse</tab>
            <frontend_type>text</frontend_type>
            <sort_order>1000</sort_order>
            <show_in_default>1</show_in_default>
            <show_in_website>1</show_in_website>
            <show_in_store>1</show_in_store>
        </techandhouse>
    </sections>
</config>

With this little snippet we have added a Tech And House tab with a sub section of Twitter. Doesn’t really do much, and when we click on the Twitter section we get an Access Denied error. This happens because our Adminhtml application can not find an entry for our new section in the ACL. Go back to the modules config.xml file and add the following (dots represent areas that we are not touching):

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        <acl>
            <resources>
                <admin>
                    <children>
                        <system>
                            <children>
                                <config>
                                    <children>
                                        <techandhouse>
                                            <title>Tech And House Module</title>
                                        </techandhouse>
                                    </children>
                                </config>
                            </children>
                        </system>
                    </children>
                </admin>
            </resources>
        </acl>
    </adminhtml>
 
    <global>
        ...
    </global>
</config>

After adding this, log out of the magento admin and log in again. Just clearing the cache will not work in this case. After you log in you will be able to click on the new module without the Access Denied error and there will be a new section in the ACL. See screenshots below:

Administration Fields

This administration panel needs some fields now. Adding options like a Yes/No dropdown and text fields can just be declared, like this:

In your system.xml file add the following under sections -> groups:

<?xml version="1.0"?>
<config>
    <tabs>
        ...
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            ...
            <groups>
                <general translate="label">
                    <label>General Options</label>
          	    <frontend_type>text</frontend_type>
          	    <sort_order>60</sort_order>
          	    <show_in_default>1</show_in_default>
          	    <show_in_website>1</show_in_website>
          	    <show_in_store>1</show_in_store>
                    <fields>
                        <enabled translate="label">
                            <label>Enable Twitter Module</label>
                            <frontend_type>select</frontend_type>
                            <source_model>adminhtml/system_config_source_yesno</source_model>
                            <sort_order>10</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </enabled>
                        <username translate="label">
                            <label>Twitter Username</label>
                            <frontend_type>text</frontend_type>
                            <sort_order>20</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </username>
                    </fields>
                </general>
            </groups>
        </techandhouse>
    </sections>
</config>

After reloading you will see your two new fields as seen in the screenshot below:

Now, if we want to give users specific options and state what our drop down options are going to be we have to do a little more work. Lets say we wanted the site administrator to set the amount of tweets to cache in the database and set the amount of time to expire the database and session caches we would perform the following steps:

Under the model folder (/app/code/local/TechAndHouse/Tweet/model/) create the following files: Database.php, Session.php and Store.php. Each file will hold options to their pertaining select box in the administration panel:

Database.php

<?php
class TechAndHouse_Tweet_Model_Database
{
    public function toOptionArray()
    {
        return array(
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),
            array('value'=>15, 'label'=>Mage::helper('tweet')->__('15')),            
            array('value'=>20, 'label'=>Mage::helper('tweet')->__('20')),          
            array('value'=>25, 'label'=>Mage::helper('tweet')->__('25')),          
            array('value'=>30, 'label'=>Mage::helper('tweet')->__('30')),          
            array('value'=>35, 'label'=>Mage::helper('tweet')->__('35')),          
            array('value'=>40, 'label'=>Mage::helper('tweet')->__('40')),          
            array('value'=>45, 'label'=>Mage::helper('tweet')->__('45')),          
            array('value'=>50, 'label'=>Mage::helper('tweet')->__('50')),          
            array('value'=>55, 'label'=>Mage::helper('tweet')->__('55')),          
            array('value'=>60, 'label'=>Mage::helper('tweet')->__('60')),                  
        );
    }
}

Session.php

<?php
class TechAndHouse_Tweet_Model_Session
{
    public function toOptionArray()
    {
        return array(
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),
            array('value'=>15, 'label'=>Mage::helper('tweet')->__('15')),            
            array('value'=>20, 'label'=>Mage::helper('tweet')->__('20')),          
            array('value'=>25, 'label'=>Mage::helper('tweet')->__('25')),          
            array('value'=>30, 'label'=>Mage::helper('tweet')->__('30')),          
            array('value'=>35, 'label'=>Mage::helper('tweet')->__('35')),          
            array('value'=>40, 'label'=>Mage::helper('tweet')->__('40')),          
            array('value'=>45, 'label'=>Mage::helper('tweet')->__('45')),          
            array('value'=>50, 'label'=>Mage::helper('tweet')->__('50')),          
            array('value'=>55, 'label'=>Mage::helper('tweet')->__('55')),          
            array('value'=>60, 'label'=>Mage::helper('tweet')->__('60')),                    
        );
    }
}

Store.php

<?php
class TechAndHouse_Tweet_Model_Store
{
    public function toOptionArray()
    {
        return array(
            array('value'=>1, 'label'=>Mage::helper('tweet')->__('1')),
            array('value'=>2, 'label'=>Mage::helper('tweet')->__('2')),
            array('value'=>3, 'label'=>Mage::helper('tweet')->__('3')),            
            array('value'=>4, 'label'=>Mage::helper('tweet')->__('4')),          
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),          
            array('value'=>6, 'label'=>Mage::helper('tweet')->__('6')),          
            array('value'=>7, 'label'=>Mage::helper('tweet')->__('7')),          
            array('value'=>8, 'label'=>Mage::helper('tweet')->__('8')),          
            array('value'=>9, 'label'=>Mage::helper('tweet')->__('9')),          
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),                     
        );
    }
}

In your modules system.xml file, add the following fields:

<?xml version="1.0"?>
<config>
    <tabs>
        ...
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            ...
            <groups>
                <general translate="label">
                    ...
                    <fields>
                         ...
                        <tweets_to_store translate="label">
                            <label>Number Of Tweets To Store In Database</label>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/store</source_model> 
                            <sort_order>30</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </tweets_to_store>
                        <session_expiry translate="label">
                            <label>Session Expires in (minutes)</label>
                            <comment>Once tweets are loaded into the session, this module will consult the Session for tweets. After the amount of minutes set here expire, the module will consult the database to see if the database has newer tweets stored. This prevents making database calls every time a page is loaded.</comment>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/session</source_model> 
                            <sort_order>40</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </session_expiry>
                        <database_expiry translate="label">
                            <label>Database cache expires in (minutes)</label>
                            <comment>Once the session is expired and a tweet is requested, the database is consulted to see if newer tweets are available. If new tweets are not available, the database copy is refreshed by calling the Twitter API and loaded into the session. Set the amount of time database sessions are valid here.</comment>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/database</source_model> 
                            <sort_order>50</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </database_expiry>
                    </fields>
                </general>
            </groups>
        </techandhouse>
    </sections>
</config>

Update (8/13/2011): Make sure your log settings are enabled in System > Developer > Log Settings for next section to work.

Now, if we reload our page we will notice that we get a blank page. To see the error, take a look at your log files. I have tail -f running on the system.log and exception.log under the /var/log/ directory at all times. If we take a look at the system.log we will notice the following error:

2010-10-01T16:58:33+00:00 ERR (3): Warning: include() [function.include]: Failed opening ‘Mage/Tweet/Model/Store.php’ for inclusion (include_path=’/Applications/MAMP/htdocs/techandhouse/app/code/local:/Applications/MAMP/htdocs/techandhouse/app/code/community:/Applications/MAMP/htdocs/techandhouse/app/code/core:/Applications/MAMP/htdocs/techandhouse/lib:.:/Applications/MAMP/bin/php5/lib/php’) in /Applications/MAMP/htdocs/techandhouse/lib/Varien/Autoload.php on line 93

But why is it looking into the Mage package when Store.php was created in the TechAndHouse package? This is because its falling back to the core to look for this file. We have not declared in our config that we are using these modules.

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <global>
      <models>
  	    <tweet>
  		    <class>TechAndHouse_Tweet_Model</class>
  		    <resourceModel>tweet_mysql4</resourceModel>
  	    </tweet>
 
            <tweet_mysql4>
                  <class>TechAndHouse_Tweet_Model_Mysql4</class>
                  <entities>
                        <tweet>
                              <table>th_tweet</table>
                        </tweet>
                  </entities>
            </tweet_mysql4>
      </models>
 
      <resources>
            ...
      </resources>
    </global>
</config>

Returning to the admin panel, we still get presented with the blank page. If we take a look at the system.log file, we notice that the error has changed. We are missing the default helper. A lot of magento code expects a default helper, so lets add this as well:

The error:
2010-10-01T17:01:13+00:00 ERR (3): Warning: include() [function.include]: Failed opening ‘Mage/Tweet/Helper/Data.php’ for inclusion (include_path=’/Applications/MAMP/htdocs/techandhouse/app/code/local:/Applications/MAMP/htdocs/techandhouse/app/code/community:/Applications/MAMP/htdocs/techandhouse/app/code/core:/Applications/MAMP/htdocs/techandhouse/lib:.:/Applications/MAMP/bin/php5/lib/php’) in /Applications/MAMP/htdocs/techandhouse/lib/Varien/Autoload.php on line 93

Under /app/code/local/TechAndHouse/Tweet/Helper add the default helper Data.php:

<?php
class TechAndHouse_Tweet_Helper_Data extends Mage_Core_Helper_Abstract
{
}

In the modules config.xml file add:

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <global>
      <helpers>
        <tweet>
          <class>TechAndHouse_Tweet_Helper</class>
        </tweet>
      </helpers>
 
      <models>
  	    ...
      </models>
 
      <resources>
            ...
      </resources>
    </global>
</config>

Now if we reload the admin page, we will see our complete configuration options:

Front Controller

Before we get this to do something remotely useful, lets add a front controller to be able to access and test some module model classes.

In the config.xml file, add the following:

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <frontend>
      <routers>
        <TechAndHouse_Tweet_TweetRouter>
          <use>standard</use>
          <args>
            <module>TechAndHouse_Tweet</module>
            <frontName>techandhouse-tweet</frontName>
          </args>
        </TechAndHouse_Tweet_TweetRouter>
      </routers>
    </frontend>
 
    <global>
        ...
    </global>
</config>

And add a new controller, IndexController.php in the /app/code/local/TechAndHouse/Tweet/controllers/ folder:

<?php
 
class TechAndHouse_Tweet_IndexController extends Mage_Core_Controller_Front_Action
{   
    public function indexAction() 
    {   
      echo "Hello World!";
    }
 
}
?>

When we go to http://127.0.0.1:8888/techandhouse-tweet we should now see a blank page with Hello World!. Now, before we continue to write the Twitter Model to get our objective done, lets go over some quick concepts on how to read/write to the database and session and how to retrive system configurations that we set up in the administration panel.

Accessing System Configurations In Admin

Change the indexAction in the IndexController to contain the following:

public function indexAction() 
{   
      $config = Mage::getStoreConfig('techandhouse/general');
 
      print_r($config);
 
      echo "Database Expiry:" . $config["database_expiry"] . "<br/>";
      echo "Session Expiry:" . $config["session_expiry"] . "<br/>";
      echo "Tweets To Store:" . $config["tweets_to_store"] . "<br/>";
}

Results:

Array
(
    [enabled] => 1
    [username] => techandhouse
    [tweets_to_store_in_db] => 5
    [tweets_to_store] => 5
    [session_expiry] => 15
    [database_expiry] => 15
)
Database Expiry:15
Session Expiry:15
Tweets To Store:5

Working with Magento Session

Change the indexAction to have the following:

public function indexAction()
{
      echo Mage::getSingleton('core/session')->getRandomVariable(); //will not return anything
      Mage::getSingleton('core/session')->setRandomVariable("hi");
      echo Mage::getSingleton('core/session')->getRandomVariable(); //will return "hi"
 
      Mage::getSingleton('core/session')->setRandomVariable(array("text"=>"hello world","var"=>"goodbye world")); //can even do arrays
      print_r(Mage::getSingleton('core/session')->getRandomVariable()); //will return array("text"=>"hello world","var"=>"goodbye world")
}

Working with Models And Database

Before we can start working with the table we created earlier in this post we need to create a model that will hold all of our business logic and the collection object in case we work with the built in collection classes.

Create Tweet.php in /app/code/local/TechAndHouse/Tweet/model with:

<?php
class TechAndHouse_Tweet_Model_Tweet extends Mage_Core_Model_Abstract
{
    public function _construct() {
      parent::_construct();
      $this->_init('tweet/tweet');
    }
}

Create Tweet.php in /app/code/local/TechAndHouse/Tweet/model/Mysql4/ with:

<?php
class TechAndHouse_Tweet_Model_Mysql4_Tweet extends Mage_Core_Model_Mysql4_Abstract
{
    public function _construct()
    {
        $this->_init('tweet/tweet', 'tweet_id');
    }
}

Create Collection.php in /app/code/local/TechAndHouse/Tweet/model/Mysql4/Tweet/ with:

<?php
class TechAndHouse_Tweet_Model_Mysql4_Tweet_Collection extends Mage_Core_Model_Mysql4_Collection_Abstract 
{
    public function _construct() {
        parent::_construct();
        $this->_init('tweet/tweet');
    }
}

The model directory structure should now look like this:

Now, go back to the IndexController.php and change the indexAction to contain:

Mage::getModel('tweet/tweet')
        ->setTwitterId("1234")
      	->setText("My new tweet!")
      	->save();

Refresh the page (http://127.0.0.1:8888/techandhouse-tweet) and you will have the following created in your database:

Next, lets try reading from the database. Change the indexAction to now contain:

var_dump(Mage::getSingleton('core/resource')
              ->getConnection('core_read')
              ->select()
              ->from(array("t"=>"th_tweet"),array("text"))
              ->where("text like '%tweet%'")
              ->limit($limit)
              ->query()
              ->fetchAll());

Output: array(1) { [0]=> array(1) { ["text"]=> string(13) “My new tweet!” } }

Need to empty the table? Try:

Mage::getSingleton('core/resource')->getConnection('core_write')->query("TRUNCATE TABLE th_tweet");

Check the database, it should now be empty.

Tweet Model – API Call and database caching

Ok, with all the concepts discussed in the section above, reading the following code should be straight forward. When getTweets is called, the function will first check against our table, th_tweet to see if we have a cached copy that has not expired. If no tweets are returned, the API call is made, result stored in the database and retured to the controller:

<?php
class TechAndHouse_Tweet_Model_Tweet extends Mage_Core_Model_Abstract
{
    public function _construct() {
      parent::_construct();
      $this->_init('tweet/tweet');
    }
 
    public function getTweets($limit=1,$username="techandhouse",$database_expiry=15,$tweets_to_store=5) {    
      $tweets = $this->getLastUnexpiredTweets($limit,$database_expiry);
 
      if(empty($tweets)):
        $tweets = $this->refreshTweets($username,$tweets_to_store,$limit);
      endif;
 
      return $tweets;
    }
 
    private function refreshTweets($username,$tweets_to_store,$limit) {
      Mage::getSingleton('core/resource')->getConnection('core_write')->query("TRUNCATE TABLE th_tweet");
 
      $tweets = json_decode(file_get_contents("http://twitter.com/statuses/user_timeline/{$username}.json?count={$tweets_to_store}"),true);
 
      $i = 0;
 
      foreach ($tweets as $tweet):
        Mage::getModel('tweet/tweet')
          ->setTwitterId($tweet['id'])
          ->setText($tweet['text'])
          ->save();
 
        if($i++ < $limit):
          $return[]['text'] = $tweet['text'];
        endif;
      endforeach;      
 
      return $return;
    }
 
    private function getLastUnexpiredTweets($limit=1,$database_expiry) {
      return Mage::getSingleton('core/resource')
                   ->getConnection('core_read')
                   ->select()
                   ->from(array("t"=>"th_tweet"),array("text"))
                   ->where("updated BETWEEN DATE_SUB(NOW(), INTERVAL {$database_expiry} MINUTE) AND NOW()")
                   ->limit($limit)
                   ->query()
                   ->fetchAll();
    }
}

Now, lets test this out. Change the indexAction to have the following code:

$config = Mage::getStoreConfig('techandhouse/general');
 
$tweet = Mage::getModel('tweet/tweet')->getTweets(1,$config["username"],$config["database_expiry"],$config["tweets_to_store"]);
print_r($tweet);
 
/*
 *Result: Array ( [0] => Array ( [text] => New blog post: Cool Off With Panasonic Inverter Air Conditioners http://bit.ly/9Pj1VY ) )
 */