Archive for the ‘Web Development’ Category

Writing a custom module in Magento

Posted on March 18th, 2011 in Magento, Web Development | 8 Comments »

There are a number of articles on the internet that explain how to write custom modules in magento but for a new developer that has never worked with Zend and has little experience coding for Magento even the simplest task can be daunting. In this article I will go over building a simple twitter module with database and session caching that is configurable via the administrative panel.

Twitters API Call

http://twitter.com/statuses/user_timeline/USERNAME.json?count=NUMBER_OF_TWEETS_TO_RETURN

This is the call to the Twitter API that retrieves the last tweets made by the specified user. Don’t ever try to throw this in one of your view templates making a call to the Twitter API every page you visit as you will only get blocked by Twitter. Read number three on the API Terms of Service. The module that I will develop here caches the result of the API call in the database and the users session. If another user visits the site before the database cache times out, the user will fetch the result from the database instead of making an unnecessary API call. Once a user fetches the result, any subsequent page that they visit will grab the result from the session until it expires. Both timeouts will be set in the admin panel of the module.

Codepools and telling magento about your new module

Magento has three codepools: core, community and local. The core codepool is dedicated for all of magentos system modules, the community codepool is for modules developed by magentos partners and the local codepool is for you as the developer to write custom code. Lets set up a new package and module for our new twitter module. Since I wrote this for my company (Tech And House) my package will be TechAndHouse and the module will be Twitter. (This module is not used on our site, just something I was playing with to document magento module creation). Set up the following folder structure under /app/code/local/:

Next, declare your shell module and its codepool. Create a file named TechAndHouse_Tweet.xml under /app/etc/modules/ with the following:

<?xml version="1.0"?>
<config>
    <modules>
        <TechAndHouse_Tweet>
            <active>true</active>
            <codePool>local</codePool>
        </TechAndHouse_Tweet>
    </modules>
</config>

SQL setup

Now that magento knows about our module, lets set up the database table that will cache the API call. Create config.xml under /app/code/local/TechAndHouse/Tweet/etc/ and define the modules setup handler:

<?xml version="1.0"?>
<config>
    <modules>
        <TechAndHouse_Tweet>
          <version>0.1.0</version>
        </TechAndHouse_Tweet>
    </modules>
 
    <global>
      <resources>
 
        <tweet_setup>
          <setup>
            <module>TechAndHouse_Tweet</module>
          </setup>
          <connection>
            <use>core_setup</use>
          </connection>
        </tweet_setup>
 
        <tweet_write>
          <connection>
            <use>core_write</use>
          </connection>
        </tweet_write>
 
        <tweet_read>
          <connection>
            <use>core_read</use>
          </connection>
        </tweet_read>
 
      </resources>
    </global>
</config>

Create install script, mysql4-install-0.1.0.php to create table th_tweet in the following directory: /app/code/local/TechAndHouse/Tweet/sql/tweet_setup/

<?php
  $installer = $this;
  $installer->startSetup();
 
  $installer->run("
    DROP TABLE IF EXISTS {$this->getTable('th_tweet')};
 
    CREATE TABLE {$this->getTable('th_tweet')} (
      `tweet_id` int(11) NOT NULL AUTO_INCREMENT,
      `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      `twitter_id` bigint(20) NOT NULL,
      `text` text NOT NULL,
      PRIMARY KEY (`tweet_id`)
    ) ENGINE=MyISAM  DEFAULT CHARSET=latin1
  ");
 
  $installer->endSetup();

With this code in place, visit any page on your store and the table will be created. What happens is that upon instantiation, magento reads the /app/etc/modules directory to get information about all its active modules. Then the config.xml file of each module is read. If a SQL setup handler is found then the appropriate version number is read after comparing to the core_resource table. If the core_resource table does not have a SQL handler listed, magento runs the installer. If the version number in the config file is greater than the version number in the core_resource table, magento runs the proper updater SQL file. For example, if the version number in the config file is 0.1.0 and in the config file the version number is 0.2.0, then magento will run the mysql4-upgrade-0.1.0-0.2.0.php.

Administration Section And ACL

Create a new file, system.xml in /app/code/local/TechAndHouse/Tweet/etc/ with the following contents:

<?xml version="1.0"?>
<config>
    <tabs>
        <techandhouse translate="label">
            <label>Tech And House Modules</label>
            <sort_order>100</sort_order>
        </techandhouse>
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            <label>Twitter</label>
            <tab>techandhouse</tab>
            <frontend_type>text</frontend_type>
            <sort_order>1000</sort_order>
            <show_in_default>1</show_in_default>
            <show_in_website>1</show_in_website>
            <show_in_store>1</show_in_store>
        </techandhouse>
    </sections>
</config>

With this little snippet we have added a Tech And House tab with a sub section of Twitter. Doesn’t really do much, and when we click on the Twitter section we get an Access Denied error. This happens because our Adminhtml application can not find an entry for our new section in the ACL. Go back to the modules config.xml file and add the following (dots represent areas that we are not touching):

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        <acl>
            <resources>
                <admin>
                    <children>
                        <system>
                            <children>
                                <config>
                                    <children>
                                        <techandhouse>
                                            <title>Tech And House Module</title>
                                        </techandhouse>
                                    </children>
                                </config>
                            </children>
                        </system>
                    </children>
                </admin>
            </resources>
        </acl>
    </adminhtml>
 
    <global>
        ...
    </global>
</config>

After adding this, log out of the magento admin and log in again. Just clearing the cache will not work in this case. After you log in you will be able to click on the new module without the Access Denied error and there will be a new section in the ACL. See screenshots below:

Administration Fields

This administration panel needs some fields now. Adding options like a Yes/No dropdown and text fields can just be declared, like this:

In your system.xml file add the following under sections -> groups:

<?xml version="1.0"?>
<config>
    <tabs>
        ...
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            ...
            <groups>
                <general translate="label">
                    <label>General Options</label>
          	    <frontend_type>text</frontend_type>
          	    <sort_order>60</sort_order>
          	    <show_in_default>1</show_in_default>
          	    <show_in_website>1</show_in_website>
          	    <show_in_store>1</show_in_store>
                    <fields>
                        <enabled translate="label">
                            <label>Enable Twitter Module</label>
                            <frontend_type>select</frontend_type>
                            <source_model>adminhtml/system_config_source_yesno</source_model>
                            <sort_order>10</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </enabled>
                        <username translate="label">
                            <label>Twitter Username</label>
                            <frontend_type>text</frontend_type>
                            <sort_order>20</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </username>
                    </fields>
                </general>
            </groups>
        </techandhouse>
    </sections>
</config>

After reloading you will see your two new fields as seen in the screenshot below:

Now, if we want to give users specific options and state what our drop down options are going to be we have to do a little more work. Lets say we wanted the site administrator to set the amount of tweets to cache in the database and set the amount of time to expire the database and session caches we would perform the following steps:

Under the model folder (/app/code/local/TechAndHouse/Tweet/model/) create the following files: Database.php, Session.php and Store.php. Each file will hold options to their pertaining select box in the administration panel:

Database.php

<?php
class TechAndHouse_Tweet_Model_Database
{
    public function toOptionArray()
    {
        return array(
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),
            array('value'=>15, 'label'=>Mage::helper('tweet')->__('15')),            
            array('value'=>20, 'label'=>Mage::helper('tweet')->__('20')),          
            array('value'=>25, 'label'=>Mage::helper('tweet')->__('25')),          
            array('value'=>30, 'label'=>Mage::helper('tweet')->__('30')),          
            array('value'=>35, 'label'=>Mage::helper('tweet')->__('35')),          
            array('value'=>40, 'label'=>Mage::helper('tweet')->__('40')),          
            array('value'=>45, 'label'=>Mage::helper('tweet')->__('45')),          
            array('value'=>50, 'label'=>Mage::helper('tweet')->__('50')),          
            array('value'=>55, 'label'=>Mage::helper('tweet')->__('55')),          
            array('value'=>60, 'label'=>Mage::helper('tweet')->__('60')),                  
        );
    }
}

Session.php

<?php
class TechAndHouse_Tweet_Model_Session
{
    public function toOptionArray()
    {
        return array(
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),
            array('value'=>15, 'label'=>Mage::helper('tweet')->__('15')),            
            array('value'=>20, 'label'=>Mage::helper('tweet')->__('20')),          
            array('value'=>25, 'label'=>Mage::helper('tweet')->__('25')),          
            array('value'=>30, 'label'=>Mage::helper('tweet')->__('30')),          
            array('value'=>35, 'label'=>Mage::helper('tweet')->__('35')),          
            array('value'=>40, 'label'=>Mage::helper('tweet')->__('40')),          
            array('value'=>45, 'label'=>Mage::helper('tweet')->__('45')),          
            array('value'=>50, 'label'=>Mage::helper('tweet')->__('50')),          
            array('value'=>55, 'label'=>Mage::helper('tweet')->__('55')),          
            array('value'=>60, 'label'=>Mage::helper('tweet')->__('60')),                    
        );
    }
}

Store.php

<?php
class TechAndHouse_Tweet_Model_Store
{
    public function toOptionArray()
    {
        return array(
            array('value'=>1, 'label'=>Mage::helper('tweet')->__('1')),
            array('value'=>2, 'label'=>Mage::helper('tweet')->__('2')),
            array('value'=>3, 'label'=>Mage::helper('tweet')->__('3')),            
            array('value'=>4, 'label'=>Mage::helper('tweet')->__('4')),          
            array('value'=>5, 'label'=>Mage::helper('tweet')->__('5')),          
            array('value'=>6, 'label'=>Mage::helper('tweet')->__('6')),          
            array('value'=>7, 'label'=>Mage::helper('tweet')->__('7')),          
            array('value'=>8, 'label'=>Mage::helper('tweet')->__('8')),          
            array('value'=>9, 'label'=>Mage::helper('tweet')->__('9')),          
            array('value'=>10, 'label'=>Mage::helper('tweet')->__('10')),                     
        );
    }
}

In your modules system.xml file, add the following fields:

<?xml version="1.0"?>
<config>
    <tabs>
        ...
    </tabs> 
 
    <sections>
        <techandhouse translate="label">
            ...
            <groups>
                <general translate="label">
                    ...
                    <fields>
                         ...
                        <tweets_to_store translate="label">
                            <label>Number Of Tweets To Store In Database</label>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/store</source_model> 
                            <sort_order>30</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </tweets_to_store>
                        <session_expiry translate="label">
                            <label>Session Expires in (minutes)</label>
                            <comment>Once tweets are loaded into the session, this module will consult the Session for tweets. After the amount of minutes set here expire, the module will consult the database to see if the database has newer tweets stored. This prevents making database calls every time a page is loaded.</comment>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/session</source_model> 
                            <sort_order>40</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </session_expiry>
                        <database_expiry translate="label">
                            <label>Database cache expires in (minutes)</label>
                            <comment>Once the session is expired and a tweet is requested, the database is consulted to see if newer tweets are available. If new tweets are not available, the database copy is refreshed by calling the Twitter API and loaded into the session. Set the amount of time database sessions are valid here.</comment>
                            <frontend_type>select</frontend_type>
                            <source_model>tweet/database</source_model> 
                            <sort_order>50</sort_order>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
                        </database_expiry>
                    </fields>
                </general>
            </groups>
        </techandhouse>
    </sections>
</config>

Update (8/13/2011): Make sure your log settings are enabled in System > Developer > Log Settings for next section to work.

Now, if we reload our page we will notice that we get a blank page. To see the error, take a look at your log files. I have tail -f running on the system.log and exception.log under the /var/log/ directory at all times. If we take a look at the system.log we will notice the following error:

2010-10-01T16:58:33+00:00 ERR (3): Warning: include() [function.include]: Failed opening ‘Mage/Tweet/Model/Store.php’ for inclusion (include_path=’/Applications/MAMP/htdocs/techandhouse/app/code/local:/Applications/MAMP/htdocs/techandhouse/app/code/community:/Applications/MAMP/htdocs/techandhouse/app/code/core:/Applications/MAMP/htdocs/techandhouse/lib:.:/Applications/MAMP/bin/php5/lib/php’) in /Applications/MAMP/htdocs/techandhouse/lib/Varien/Autoload.php on line 93

But why is it looking into the Mage package when Store.php was created in the TechAndHouse package? This is because its falling back to the core to look for this file. We have not declared in our config that we are using these modules.

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <global>
      <models>
  	    <tweet>
  		    <class>TechAndHouse_Tweet_Model</class>
  		    <resourceModel>tweet_mysql4</resourceModel>
  	    </tweet>
 
            <tweet_mysql4>
                  <class>TechAndHouse_Tweet_Model_Mysql4</class>
                  <entities>
                        <tweet>
                              <table>th_tweet</table>
                        </tweet>
                  </entities>
            </tweet_mysql4>
      </models>
 
      <resources>
            ...
      </resources>
    </global>
</config>

Returning to the admin panel, we still get presented with the blank page. If we take a look at the system.log file, we notice that the error has changed. We are missing the default helper. A lot of magento code expects a default helper, so lets add this as well:

The error:
2010-10-01T17:01:13+00:00 ERR (3): Warning: include() [function.include]: Failed opening ‘Mage/Tweet/Helper/Data.php’ for inclusion (include_path=’/Applications/MAMP/htdocs/techandhouse/app/code/local:/Applications/MAMP/htdocs/techandhouse/app/code/community:/Applications/MAMP/htdocs/techandhouse/app/code/core:/Applications/MAMP/htdocs/techandhouse/lib:.:/Applications/MAMP/bin/php5/lib/php’) in /Applications/MAMP/htdocs/techandhouse/lib/Varien/Autoload.php on line 93

Under /app/code/local/TechAndHouse/Tweet/Helper add the default helper Data.php:

<?php
class TechAndHouse_Tweet_Helper_Data extends Mage_Core_Helper_Abstract
{
}

In the modules config.xml file add:

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <global>
      <helpers>
        <tweet>
          <class>TechAndHouse_Tweet_Helper</class>
        </tweet>
      </helpers>
 
      <models>
  	    ...
      </models>
 
      <resources>
            ...
      </resources>
    </global>
</config>

Now if we reload the admin page, we will see our complete configuration options:

Front Controller

Before we get this to do something remotely useful, lets add a front controller to be able to access and test some module model classes.

In the config.xml file, add the following:

<?xml version="1.0"?>
<config>
    <modules>
        ...
    </modules>
 
    <adminhtml>
        ...
    </adminhtml>
 
    <frontend>
      <routers>
        <TechAndHouse_Tweet_TweetRouter>
          <use>standard</use>
          <args>
            <module>TechAndHouse_Tweet</module>
            <frontName>techandhouse-tweet</frontName>
          </args>
        </TechAndHouse_Tweet_TweetRouter>
      </routers>
    </frontend>
 
    <global>
        ...
    </global>
</config>

And add a new controller, IndexController.php in the /app/code/local/TechAndHouse/Tweet/controllers/ folder:

<?php
 
class TechAndHouse_Tweet_IndexController extends Mage_Core_Controller_Front_Action
{   
    public function indexAction() 
    {   
      echo "Hello World!";
    }
 
}
?>

When we go to http://127.0.0.1:8888/techandhouse-tweet we should now see a blank page with Hello World!. Now, before we continue to write the Twitter Model to get our objective done, lets go over some quick concepts on how to read/write to the database and session and how to retrive system configurations that we set up in the administration panel.

Accessing System Configurations In Admin

Change the indexAction in the IndexController to contain the following:

public function indexAction() 
{   
      $config = Mage::getStoreConfig('techandhouse/general');
 
      print_r($config);
 
      echo "Database Expiry:" . $config["database_expiry"] . "<br/>";
      echo "Session Expiry:" . $config["session_expiry"] . "<br/>";
      echo "Tweets To Store:" . $config["tweets_to_store"] . "<br/>";
}

Results:

Array
(
    [enabled] => 1
    [username] => techandhouse
    [tweets_to_store_in_db] => 5
    [tweets_to_store] => 5
    [session_expiry] => 15
    [database_expiry] => 15
)
Database Expiry:15
Session Expiry:15
Tweets To Store:5

Working with Magento Session

Change the indexAction to have the following:

public function indexAction()
{
      echo Mage::getSingleton('core/session')->getRandomVariable(); //will not return anything
      Mage::getSingleton('core/session')->setRandomVariable("hi");
      echo Mage::getSingleton('core/session')->getRandomVariable(); //will return "hi"
 
      Mage::getSingleton('core/session')->setRandomVariable(array("text"=>"hello world","var"=>"goodbye world")); //can even do arrays
      print_r(Mage::getSingleton('core/session')->getRandomVariable()); //will return array("text"=>"hello world","var"=>"goodbye world")
}

Working with Models And Database

Before we can start working with the table we created earlier in this post we need to create a model that will hold all of our business logic and the collection object in case we work with the built in collection classes.

Create Tweet.php in /app/code/local/TechAndHouse/Tweet/model with:

<?php
class TechAndHouse_Tweet_Model_Tweet extends Mage_Core_Model_Abstract
{
    public function _construct() {
      parent::_construct();
      $this->_init('tweet/tweet');
    }
}

Create Tweet.php in /app/code/local/TechAndHouse/Tweet/model/Mysql4/ with:

<?php
class TechAndHouse_Tweet_Model_Mysql4_Tweet extends Mage_Core_Model_Mysql4_Abstract
{
    public function _construct()
    {
        $this->_init('tweet/tweet', 'tweet_id');
    }
}

Create Collection.php in /app/code/local/TechAndHouse/Tweet/model/Mysql4/Tweet/ with:

<?php
class TechAndHouse_Tweet_Model_Mysql4_Tweet_Collection extends Mage_Core_Model_Mysql4_Collection_Abstract 
{
    public function _construct() {
        parent::_construct();
        $this->_init('tweet/tweet');
    }
}

The model directory structure should now look like this:

Now, go back to the IndexController.php and change the indexAction to contain:

Mage::getModel('tweet/tweet')
        ->setTwitterId("1234")
      	->setText("My new tweet!")
      	->save();

Refresh the page (http://127.0.0.1:8888/techandhouse-tweet) and you will have the following created in your database:

Next, lets try reading from the database. Change the indexAction to now contain:

var_dump(Mage::getSingleton('core/resource')
              ->getConnection('core_read')
              ->select()
              ->from(array("t"=>"th_tweet"),array("text"))
              ->where("text like '%tweet%'")
              ->limit($limit)
              ->query()
              ->fetchAll());

Output: array(1) { [0]=> array(1) { ["text"]=> string(13) “My new tweet!” } }

Need to empty the table? Try:

Mage::getSingleton('core/resource')->getConnection('core_write')->query("TRUNCATE TABLE th_tweet");

Check the database, it should now be empty.

Tweet Model – API Call and database caching

Ok, with all the concepts discussed in the section above, reading the following code should be straight forward. When getTweets is called, the function will first check against our table, th_tweet to see if we have a cached copy that has not expired. If no tweets are returned, the API call is made, result stored in the database and retured to the controller:

<?php
class TechAndHouse_Tweet_Model_Tweet extends Mage_Core_Model_Abstract
{
    public function _construct() {
      parent::_construct();
      $this->_init('tweet/tweet');
    }
 
    public function getTweets($limit=1,$username="techandhouse",$database_expiry=15,$tweets_to_store=5) {    
      $tweets = $this->getLastUnexpiredTweets($limit,$database_expiry);
 
      if(empty($tweets)):
        $tweets = $this->refreshTweets($username,$tweets_to_store,$limit);
      endif;
 
      return $tweets;
    }
 
    private function refreshTweets($username,$tweets_to_store,$limit) {
      Mage::getSingleton('core/resource')->getConnection('core_write')->query("TRUNCATE TABLE th_tweet");
 
      $tweets = json_decode(file_get_contents("http://twitter.com/statuses/user_timeline/{$username}.json?count={$tweets_to_store}"),true);
 
      $i = 0;
 
      foreach ($tweets as $tweet):
        Mage::getModel('tweet/tweet')
          ->setTwitterId($tweet['id'])
          ->setText($tweet['text'])
          ->save();
 
        if($i++ < $limit):
          $return[]['text'] = $tweet['text'];
        endif;
      endforeach;      
 
      return $return;
    }
 
    private function getLastUnexpiredTweets($limit=1,$database_expiry) {
      return Mage::getSingleton('core/resource')
                   ->getConnection('core_read')
                   ->select()
                   ->from(array("t"=>"th_tweet"),array("text"))
                   ->where("updated BETWEEN DATE_SUB(NOW(), INTERVAL {$database_expiry} MINUTE) AND NOW()")
                   ->limit($limit)
                   ->query()
                   ->fetchAll();
    }
}

Now, lets test this out. Change the indexAction to have the following code:

$config = Mage::getStoreConfig('techandhouse/general');
 
$tweet = Mage::getModel('tweet/tweet')->getTweets(1,$config["username"],$config["database_expiry"],$config["tweets_to_store"]);
print_r($tweet);
 
/*
 *Result: Array ( [0] => Array ( [text] => New blog post: Cool Off With Panasonic Inverter Air Conditioners http://bit.ly/9Pj1VY ) )
 */

Setting up Magento on Rackspace CouldServer

Posted on December 16th, 2010 in Magento, Ubuntu, Web Development | No Comments »

Objective: Prep a test server for a Magento installation on Ubuntu 10.10

In this example I spun up a quick server on Rackspace that has the bare minimums of Ubuntu installed. So everything needs to get installed. Before getting started, take a look at the server requirements to run Magento and run the magento check utility before attempting to install magento to make sure all required components are installed.

Step 1: Update apt-get

sudo apt-get update

Step 2: Install updatedb. Not necessary, just like having it around.

sudo apt-get install locate

Step 3: Install PHP 5

sudo apt-get install php5

Step 4: Install Apache 2.2

sudo apt-get install apache2

Step 5: Enable Apache Mod Rewrite and restart Apache

sudo a2enmod rewrite
sudo /etc/init.d/apache2 restart

Step 6: Install MySQL 5

sudo apt-get install mysql-server

Step 7: Install required PHP extensions and restart Apache

sudo apt-get install php5-mysql php5-mcrypt php5-gd php5-curl php-pear php5-dev libmysqlclient-dev libcurl3-openssl-dev
sudo /etc/init.d/apache2 restart

Step 8: Setup a virtualhost for the new test site Further Reading

sudo cp /etc/apache2/sites-available/default /etc/apache2/sites-available/techandhouse_dev
sudo vi /etc/apache2/sites-available/techandhouse_dev
#Change the DocumentRoot to point to the new location. Example:
/home/user/techecom_dev/
#Change the Directory directive, replace:
<Directory /var/www/> to <Directory /home/user/techecom_dev/>

Sample config:

<!--Change AllowOverride to All for our site for mod rewrite to work-->
<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName techandhouse_dev
 
        DocumentRoot /home/techecom_dev/public_html/
        <Directory />
                Options FollowSymLinks
                AllowOverride All
        </Directory>
        <Directory /home/techecom_dev/public_html/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
 
        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
        </Directory>
 
        ErrorLog ${APACHE_LOG_DIR}/error.log
 
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
 
        CustomLog ${APACHE_LOG_DIR}/access.log combined
 
    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>
 
</VirtualHost>

Deactivate the old site, activate the new one, and restart Apache.

sudo a2dissite default && sudo a2ensite techandhouse_dev
sudo /etc/init.d/apache2 restart

If you run the magento check utility and go to the IP address or domain of your machine, you should see:

Follow instructions here to move the web site to the test server…

After moving the web site, update a couple values so that the site doesn’t route itself back to the real domain because of the database or cached files:

update core_config_data set value="http://DOMAIN/" where path=’web/unsecure/host’;     
update core_config_data set value="http://DOMAIN/" where path=’web/secure/host’;
cd /path/to/your/magento/site
rm -r var/cache/* var/session/*

Magento: Custom attributes for customer profiles

Posted on November 30th, 2010 in Magento, Web Development | 1 Comment »

If you’re looking to synchronize your customer, sales order and invoices between your back office system and Magento you’re going to need to store the back offices unique ID within Magento. Unlike products, adding custom attributes and attribute sets is not so straight forward. You can not just add it within the administration interface.

Task: Add field erp_customer_id to customer profile (same concept applies to sales orders and invoices)

Step 1: Let Magento know we are creating a new module and add TechAndHouse_Customer.xml in /app/etc/modules/ with the following:

<config>
	<modules>
		<TechAndHouse_Customer>
			<active>true</active>
			<codePool>local</codePool>
		</TechAndHouse_Customer>
	</modules>
</config>

Step 2: Create a new module under /app/code/local/ with the following directory structure:

Step 3a: Copy the contents of /app/code/core/Mage/Customer/etc/config.xml into /app/code/local/TechAndHouse/Customer/etc/config.xml and change:

<modules>
        <Mage_Customer>
            <version>x.x.x</version>
        </Mage_Customer>
</modules>

to

<modules>
        <TechAndHouse_Customer>
            <version>1.0.0</version>
        </TechAndHouse_Customer>
</modules>

Step 3b: In this same file, config.xml add the variable erp_customer_id into the global fieldset under customer_account

<erp_customer_id><create>1</create><update>1</update></erp_customer_id>

Step 4a: Copy the getDefaultEntities method from /app/code/core/Mage/Customer/Model/Entity/Setup.php file to /app/code/local/TechAndHouse/Model/Entity/Setup.php and put it in the following class:

class TechAndHouse_Customer_Model_Entity_Setup extends Mage_Customer_Model_Entity_Setup {

Step 4b: Add the following attribute in the array right before website_id

'erp_customer_id' => array(
                  	'label'		=> 'ERP Customer ID',
                  	'visible'	=> true,
                  	'required'	=> true,
                  ),

Step 5: Add the following in any theme file and visit the page, like /template/customer/form/register.phtml to create the attribute in the database:

$setup = new TechAndHouse_Customer_Model_Entity_Setup('core_setup');
$setup->addAttribute('customer', 'erp_customer_id', array(
	'label'		=> 'ERP Customer ID',
	'type'		=> 'varchar',
	'input'		=> 'text',
	'visible'	=> true,
	'required'	=> true,
	'position'	=> 1,
	));

Step 6: Check if the variable got created in the eav_attribute table.

Step 7: Did it work? At this point you can run a quick test and see if you can write and read from your new attribute. Never modify core files, and do not do anything on a production machine. The following was done on a development box.

Add the following to /design/adminhtml/default/default/template/customer/tab/view.phtml:

$this->getCustomer()->setErpCustomerId("1234");
print_r($this->getCustomer()->getErpCustomerId());

Visit a customer in the admin section and you will see the following:

Credit

Original Article: Fontis

CakePHP: Session value is favicon.ico, Why?

Posted on June 28th, 2010 in Web Development | No Comments »

It was time to give CakePHPs Auth component a little test drive. Thinking I followed the manual to the key, there was no way I missed a step. I mean, there isn’t really much to do to set up simple authentication using CakePHP right? Well, for some reason after logging in I would get pushed to http://localhost:8888/users/favicon.ico:

Anyway, googling, asking for help on #cakephp didn’t really help as the problem was so stupid, not many people have had this problem. I stumbled across this post on the CakePHP google group.

Basically – “make sure the link to favicon.ico is root-relative (/favicon.ico). Otherwise the server’s looking for /controller/action/favicon.ico and the broken link gets captured as the current page. Same thing for your css and js.” – Chris Cassell

(Thank you sir!)

CakePHP: Autotest using Watchr

Posted on May 9th, 2010 in Web Development | 4 Comments »

I was taking a look at ZenTest a while ago, especially the autotest functionality. As stated in the ZenTest site, autotest:

  • Improves feedback by running tests continuously.
  • Continually runs tests based on files you’ve changed.
  • Get feedback as soon as you save. Keeps you in your editor allowing you to get stuff done faster.

To add more bang to your buck, you can get continuous notifications from growl every time you save a file, alerting you if any of your previous test cases have failed.

Googling around I wanted to see if this can be tied into testing with CakePHP. If I did a horrible job googling and there is already something out there that mimics this functionality please send me a link. Anyways, I stumbled across Watchr. Watchr is a tool that monitors a directory tree, and triggers a user defined action whenever an observed file is modified. At that point trigger a CakePHP test suite command line argument and send the results to growl. And there we have it, continuous testing with growl notifications on CakePHP using Watchr.

Step 1: Install the Watchr gem: http://github.com/mynyml/watchr

Step 2: Upload images that growl will use to display passed/failed cases

Step 3: Set up a watchr configuration file in app/config/watchr.rb

Step 4: Every time you develop have watchr runing. Type the following in your root directory:

watchr app/config/watchr.rb

End Result:

Pass:

Fail:

Failed and Passed images to pass to growl

Create a directory ~/.watchr_images and upload the following images or any image of your choice for failed and passed test cases:

Break down of the watchr.rb configuration file

Define a global variable that points to your cake console script. I wanted to load this variable from my aliases in .profile but couldn’t figure out how (my ruby skills suck):

$cake = "/Applications/MAMP/bin/php5/bin/php /Applications/MAMP/htdocs/yourProject/cake/console/cake.php"

Define a function that will parse out the details you want to pass to growl. I only cared about the number of passes and fails.

def growl(message)
  message = message.split('---------------------------------------------------------------')[3].split('Time taken by tests')[0]
  growlnotify = `which growlnotify`.chomp
  image = message.include?('fails') ? "~/.watchr_images/failed.png" : "~/.watchr_images/passed.png"
  options = "-w -n Watchr --image '#{File.expand_path(image)}' -m '#{message}'"
  system %(#{growlnotify} #{options} &)
end

Define a function that will run a command-line program and passes the result to growl

def run(cmd) 
  puts(cmd)
  result = `#{cmd}`
  growl result rescue nil
  puts result
end

Define a function that calls the test case of the pertaining model or controller that just got saved:

def test_changed_model_or_controller(file)
  type = file.split('/')[1]
  name = file.split('/')[2].split('.')[0]
  run "#{$cake} testsuite app case #{type}/#{name}" 
end

Define a function that calls the test case of the pertaining test case that just got saved:

def test_changed_test_case(file)
  type = file.split('/')[3]
  name = file.split('/')[4].split('.')[0]
  run "#{$cake} testsuite app case #{type}/#{name}"
end

Define a function that runs the entire test suite for changes in config files, app_controller and app_model:

def test_app()
  run "#{$cake} testsuite app all"
end

The heart of the script, the watchers. Here we state which files to monitor using regular expression pattern matching paths.

watch('app/(models|controllers)/(.*).php')  { |m| test_changed_model_or_controller(m[0]) }
watch('app/tests/cases/(models|controllers)/(.*).test.php')  { |m| test_changed_test_case(m[0]) }
watch('app/config/(.*).php') { |m| test_app() }
watch('app/(app_controller|app_model).php')  { |m| test_app() }

That’s it. Run the watcher via “watchr app/config/watchr.rb” as you develop and your test cases will continuously run giving you growl update notifications.

Download the watchr.rb config file


Sources:
Watchr Readme
Watchr – Most of my time was spent here modifying this script and applying it to a CakePHP scenario
Setting up Watchr and Rails
Watchr: A Flexible, Generic Alternative to AutoTest – This was the first article I stumbled across that got me started on Watchr
CakePHP – Running Tests in the Command Line

Web Scraping using PHP and XPath

Posted on March 30th, 2010 in Web Development | 1 Comment »

Yet another thing on my to-do list has been to either get someone to either start updating the product catalog on the Tech And House site, or to get a scraper to extract the information on the Frigidaire site. Didn’t realize how easy it was to whip up an extraction script using XPath! First, if you don’t already have the XPather add-on for Firefox, download it. Makes determining the XPath to elements on the page very easy:



Using the code below, scraping data from pretty much any site shouldn’t be too difficult. Modifying the XPaths in frigidaire_scraper.php and its extract_specs() method are where the main changes will occur to build a scraper for another site.

File 1: scraper.php

<?php
class Scraper {
  var $oldSetting;
  var $html;
 
  function scraper($targetUrl) {
    $this->oldSetting = libxml_use_internal_errors( true ); 
    libxml_clear_errors(); 
 
    $this->html = new DOMDocument(); 
    $this->html->loadHtmlFile($targetUrl); 
  }
 
  function extract($links_xpath,$extract,$include_blanks=true) {
    $return = array();
 
    $xpath = new DOMXPath($this->html); 
    $items = $xpath->query($links_xpath);
 
    foreach ($items as $item) {        
    	$newDom = new DOMDocument;
    	$newDom->appendChild($newDom->importNode($item,true));
 
    	$xpath = new DOMXPath($newDom); 
    	$extraction = trim($xpath->query($extract)->item(0)->nodeValue);
 
    	if ($include_blanks==true)
    	  array_push($return,$extraction);
    	else if ($extraction!="")
    	  array_push($return,$extraction);
    }
 
    return $return;
  }
}
?>

File 2: frigidaire_scraper.php

<?php
require ("scraper.php");
 
class FrigidaireScraper extends Scraper {
 
  var $category_links;
  var $product_links;
  var $products;
  var $keys;
 
  function FrigidaireScraper($targetUrl) {
    parent::scraper($targetUrl);
  }
 
  function extract_category_links() {
    $this->category_links = $this->extract("//div[@id='left-nav']/a[@class='left-nav-item-sub']","@href");
  }
 
  function extract_product_links() {
    $this->product_links = array();
    foreach($this->category_links as $link): 
      parent::scraper($link);
      $products = $this->extract("//table[@id='ProductTable']//h4/a","@href");
      $this->product_links = array_merge($this->product_links,$products);
    endforeach;
  }
 
  function extract_specs() {
    $this->keys = array();
    $this->products = array();
 
    foreach($this->product_links as $link):  
      parent::scraper($link);
      $model = explode(": ", array_pop($this->extract("//div[@id='main-inner']//h5","text()[1]")));
      $model = $model[1];
      $msrp = explode(" ", array_pop($this->extract("//div[@id='main-inner']//h5","text()[3]")));
      $msrp = $msrp[1];
 
      $specs = $this->extract("//div[@id='ctl00_CPHMain_TabContainer1_TabSpecifications']/div[2]/div/div[1]/div","text()",false);
 
      $this->products[$model]["Price"] = $msrp;
      $this->products[$model]["Model"] = $model;
      $this->products[$model]["Link"] = $link;
 
      foreach($specs as $spec):
        $spec = explode(": ", $spec);
        $this->products[$model][$spec[0]] = $spec[1];
      endforeach;
 
      $this->keys = array_unique(array_merge($this->keys,array_keys($this->products[$model])));
    endforeach;
  }
}
?>

File 3: scrape_it.php (bring it all together)

<?php
require_once("frigidaire_scraper.php");
 
$scraper = new FrigidaireScraper("http://www.frigidaire.com");
$scraper->extract_category_links();
$scraper->extract_product_links();
$scraper->extract_specs();
 
//echo a simple tab delimited file structure
foreach ($scraper->keys as $key):
  echo $key."\t";
endforeach;
 
echo "\n";
 
foreach ($scraper->products as $row):    
  foreach ($scraper->keys as $key):
    if (isset($row[$key]))
      echo $row[$key] . "\t";
    else echo "\t";
  endforeach;
  echo "\n";
endforeach;
?>

Scrape Output:

Installing ImageMagicK and its dependencies

Posted on February 8th, 2010 in Web Development | No Comments »

To get barcode-generator for rails working I needed to get ImageMagicK installed. I had some problems installing it using the instructions on the site. I was using the following posts to get ImageMagicK installed:

RMagicK from source on Snow Leopard
Installing ImageMagicK & RmagicK on Leopard
RMagick & ImageMagick from Source: Mac OSX Leopard

Problem 1: For some reason curl was not working. I could not decompress the files. When attempting to decompress, I would get a file with an extension of .cpgz. Attempting to open that file, I would get a zip file, unzipping that file I would get a .cpgz file. Still don’t know what was up with that! Vicious never ending cycle. Other people with similar issue.

Problem 2: Could not get littlecms and libwmf compiled and installed. Would fail at the make clean step.

Problem 3: ImageMagicK installation instructions using the Apple binary did not work for me.

Below you can find the steps I used to get ImageMagicK compiled and installed with updated links to the latest stable releases – as of today:

wget http://hivelocity.dl.sourceforge.net/project/freetype/freetype2/2.3.11/freetype-2.3.11.tar.gz
tar xzvf freetype-2.3.11.tar.gz
cd freetype-2.3.11
./configure --prefix=/usr/local
make
sudo make install
cd ..
 
wget ftp://ftp.simplesystems.org/pub/libpng/png/src/libpng-1.4.0.tar.gz
tar xzvf libpng-1.4.0.tar.gz
cd libpng-1.4.0
./configure --prefix=/usr/local
make
sudo make install
cd ..
 
wget http://www.ijg.org/files/jpegsrc.v8.tar.gz
tar xzvf jpegsrc.v8.tar.gz
cd jpeg-8
ln -s `which glibtool` ./libtool
export MACOSX_DEPLOYMENT_TARGET=10.6.2
./configure --enable-shared --prefix=/usr/local
make
sudo make install
cd ..
 
wget ftp://ftp.remotesensing.org/pub/libtiff/tiff-3.9.2.tar.gz
tar xzvf tiff-3.9.2.tar.gz
cd tiff-3.9.2
./configure --prefix=/usr/local
make
sudo make install
cd ..
 
wget http://ghostscript.com/releases/ghostscript-8.70.tar.gz
tar xzvf ghostscript-8.70.tar.gz
cd ghostscript-8.70/
./configure  --prefix=/usr/local
make
sudo make install
cd ..
 
wget ftp://ftp.imagemagick.org/pub/ImageMagick/delegates/ghostscript-fonts-std-8.11.tar.gz
tar xzvf ghostscript-fonts-std-8.11.tar.gz
sudo mv fonts /usr/local/share/ghostscript
 
wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar xzvf ImageMagick.tar.gz
cd ImageMagick-6.5.9-3/
export CPPFLAGS=-I/usr/local/include
export LDFLAGS=-L/usr/local/lib
./configure --prefix=/usr/local --disable-static --with-modules --without-perl --without-magick-plus-plus --with-quantum-depth=8 --with-gs-font-dir=/usr/local/share/ghostscript/fonts
make
sudo make install
cd ..

After installation, check if ImageMagicK was installed correctly. The following command should create a new file called logo.gif with the ImageMagicK logo:

/usr/local/bin/convert logo: logo.gif

For a more comprehensive test, run the following command:

make check

Results:

make check
make  check-am
make  tests/validate  wand/drawtest wand/wandtest
  CC     tests/tests_validate-validate.o
  CCLD   tests/validate
  CC     wand/drawtest.o
  CCLD   wand/drawtest
  CC     wand/wandtest.o
  CCLD   wand/wandtest
make  check-TESTS check-local
PASS: tests/validate-compare.sh
PASS: tests/validate-composite.sh
PASS: tests/validate-convert.sh
PASS: tests/validate-formats-on-disk.sh
PASS: tests/validate-formats-in-memory.sh
PASS: tests/validate-identify.sh
PASS: tests/validate-import.sh
PASS: tests/validate-montage.sh
PASS: tests/validate-stream.sh
PASS: wand/drawtest.sh
PASS: wand/wandtest.sh
===================
All 11 tests passed
===================

MySQL Gem on 64-bit Snow Leopard

Posted on November 28th, 2009 in Web Development | No Comments »

Finally took some time to move a small rails application from my old Macbook Pro to my Macbook Pro running Snow Leopard and MAMP. Setting up autotest which should have just been a few commands ended up being a battle getting the MySQL gem set up correctly.

Autotest setup

What should have been quick and simple…

sudo gem install ZenTest
gem install autotest-rails

Make a file named .autotest in the root directory of the application and add:

require "autotest/growl"

Ended up being…

1) Update the system first.:

sudo gem update --system

Error:

!!! The bundled mysql.rb driver has been removed from Rails 
2.2. Please install the mysql gem and try again: gem install 
mysql.

From this point the battle starts. I have recompiled the MySQL gem many times trying to get it working. Excerpts of a few errors:

Enclosing class/module 'mXML' for class XPointer not known
ERROR:  Error installing mysql:
ERROR: Failed to build gem native extension.
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lmygcc... no
checking for mysql_query() in -lmysqlclient... no
Gem files will remain installed in /Library/Ruby/Gems/1.8/gems/mysql-2.8.1 for inspection.
Results logged to /Library/Ruby/Gems/1.8/gems/mysql-2.8.1/ext/mysql_api/gem_make.out
 
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README
ERROR:  Error installing rubynode:
ERROR: Failed to build gem native extension.
 
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb
==================== ERROR =====================
Please set RUBY_SOURCE_DIR to the source path of ruby 1.8.7 (2008-08-11)!
================================================
uninitialized constant MysqlCompat::MysqlRes

The solution…

Finally got it working following the steps in this article. One gotcha of this article – since we are installing on a 64 bit operating system the last step needs to be installed with the -arch x86_64 flag, not -arch i386.

First download the MAMP source here, then:

unzip MAMP_1.7.2_src.zip
cd MAMP_1.7.2_src
tar -xzvf mysql-5.0.41.tar.gz
cd mysql-5.0.41 
./configure --with-unix-socket-path=/Applications/MAMP/tmp/mysql/mysql.sock --without-server --prefix=/Applications/MAMP/Library
make -j2
cp libmysql/.libs/*.dylib /Applications/MAMP/Library/lib/mysql 
mkdir /Applications/MAMP/Library/include
cp -R include /Applications/MAMP/Library/include/mysql
sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/Applications/MAMP/Library/bin/mysql_config

Git – Ignore already checked in file/directory

Posted on November 14th, 2009 in Web Development | No Comments »

Original Source: stackoverflow.com

This command will cause git to untrack your directory and all files under it without actually deleting them:

git rm -r --cached <your directory>

The -r option causes the removal of all files under your directory.
The –cached option causes the files to only be removed from git’s index, not your working copy. By default git rm would delete .

Renaming files – Adding a suffix to the file name

Posted on October 23rd, 2009 in Web Development | 1 Comment »

The solution provided in Easily Renaming Multiple Files works perfectly well if we want to add a prefix to a file. This solution does not work too well if you want to add a suffix.

Objective: For all jpegs in a folder add the suffix _tn after the file name and before the extension.
Example: file1.jpg = file1_tn.jpg

In the console type the following:

for i in *.jpg ; do mv $i `echo $i | sed 's/\.jpg/\_tn.jpg/'` ; done

If the file name was in CAPS and and you’d rather have your file name in lowercase pipe the following on:

for i in *.jpg ; do mv $i `echo $i | sed 's/\.jpg/\_tn.jpg/' | tr [:upper:] [:lower:]` ; done