Caching using PHP/Zend_Cache and MySQL

I like the definition used in Wikipedia: “a cache is a temporary storage area where often accessed data can be stored for quick access”. The idea is to get ‘often accessed data’ from a database and store it in memory (RAM or as a file in your local file system). This is because:

  • it’s quicker for a machine to read from memory than to connect to a database and query data.
  • it’s more efficient for the database to not waste time and resources returning the same dataset multiple times when it could be focusing on other tasks.

As long as the data, in this scenario from the database, doesn’t change, there is no need to query it again.

Resources are limited on systems and to take advantage of your resources, you need to make sure time isn’t spent on tasks that could be handled better elsewhere. Here is a silly real world example. Imagine on a daily basis, I have to track how many magazines I have and send this information to Person X. I get new magazines at the beginning of each month only. To track the number of magazines I have every day I could

  1. Count them, one by one every day and send Person X the total. If I have 50 magazines this could take some time and assume I get 10 more every month, after a year or two I could spend all day just counting how many magazines I have instead of working. Sound productive?
  2. Count them once and write the number down on a piece of paper (caching!). Everyday when Person X asks how many magazines I have, I read the number from the piece of paper. Only when I get new magazines (once a month) do I count them again (or just add the current number + the new amount) to get my new total. Then I update my piece of paper with the new total (updating the value in cache).

The latter is definitely the more productive choice.

The same idea applies to computer systems. In the web, you have static and dynamic files. Static files are quicker to serve on a server because the server only has to read the contents of the file and send it to the browser requesting it. Dynamic pages take more time and resources because the server needs to execute the code in the page and only once it’s done can it send the request back. PHP can be used to create dynamic pages. The server executes the php code and spits out a file that then is read by the browser. If a database is involved, then the database has to run it’s task as well before the final file is returned.

When ever possible, it’s more efficient to serve a static file or static content. We use cache to accomplish this. In this post I’m going to talk about caching files and database queries to local files on the server.

Zend_Cache

There are different ways to achieve this. I personally use Zend Framework on my projects so I’ll be using Zend_Cache in my examples. I will only be using Zend_Cache as a standalone module, not the entire framework. This way, those of you who don’t use Zend Framework can still follow this guide. There are other options if you don’t have Zend such as Cache_Lite which is part of the PEAR Framework. Both work very similarly.

Zend_Cache is very flexible in that it lets you decide what you want to cache (the frontend) and where you want to put it (the backend). The different frontends for Zend_Cache include (taken from the Zend docs):

  • Zend_Cache_Core is a special frontend because it is the core of the module. It is a generic cache frontend and is extended by other classes.
  • Zend_Cache_Frontend_Output is an output-capturing frontend. It utilizes output buffering in PHP to capture everything between its start() and end() methods.
  • Zend_Cache_Frontend_Function caches the results of function calls. It has a single main method named call() which takes a function name and parameters for the call in an array.
  • Zend_Cache_Frontend_Class is different from Zend_Cache_Frontend_Function because it allows caching of object and static method calls.
  • Zend_Cache_Frontend_File is a frontend driven by the modification time of a “master file”. It’s really interesting for examples in configuration or templates issues. It’s also possible to use multiple master files.
  • Zend_Cache_Frontend_Page is like Zend_Cache_Frontend_Output but designed for a complete page. It’s impossible to useZend_Cache_Frontend_Page< for caching only a single block.

The backends include:

  • Zend_Cache_Backend_File – This (extended) backends stores cache records into files (in a choosen directory).
  • Zend_Cache_Backend_Sqlite – This (extended) backends stores cache records into a SQLite database.
  • Zend_Cache_Backend_Memcached – This (extended) backends stores cache records into a memcached server. memcached is a high-performance, distributed memory object caching system. To use this backend, you need a memcached daemon and the memcache PECL extension.
  • Zend_Cache_Backend_Apc – This (extended) backends stores cache records in shared memory through the APC (Alternative PHP Cache) extension (which is of course need for using this backend).
  • Zend_Cache_Backend_Xcache – This backends stores cache records in shared memory through the XCache extension (which is of course need for using this backend).
  • and a couple more you can check in the Zend docs.

For my first example I’ll be using the ‘Core’ frontend (to cache a variable) and the ‘File’ backend (to save that variable to a file on the server). I will actually be using the ‘File’ backend on all my examples since I have not had the opportunity to work with and of the other backend methods. Reading from RAM is quicker than reading from the file system so using other backend methods which take advantage of this would yield better results.

Setting up the Environment

Before I go into the first example, let me explain how I set up my environment. Like i mentioned earlier, I won’t be using the Zend Framework, instead I will only be using Zend_Cache as a standalone module. To accomplish this, I create a library folder in my site root. Then I create a Zend folder inside of library where I will be putting the Cache module in. The next step requires that you download the Zend Framework zip file so you can copy the cache module found in the zip (minimal package is all you need). Once you download the file, open it and copy Cache.php from /library/Zend/Cache.php and the /library/Zend/Cache folder to your Zend folder in library. Here is an image of what your structure should look like:

Folder Structure
Folder Structure

Example 1 – Caching a Variable

The first example is a slightly modified version of the example given on the Zend_Cache docs:

<?php
include 'library/Zend/Cache.php';

$frontendOptions = array(
   'lifetime' => 10,
   'automatic_serialization' => true
);

$backendOptions = array(
    'cache_dir' => 'tmp/'
);

$cache = Zend_Cache::factory('Core', 'File', $frontendOptions, $backendOptions);
$id = 'myBigLoop';

$start_time = microtime(true);

if(!($data = $cache->load($id)))
{
    echo "Not found in Cache<br />";

    $data = '';
    for ($i = 0; $i < 1000000; $i++)
    {
        $data = $data . $i;
    }
    $cache->save($data);
}
else
{
	echo "Running from Cache<br />";
}

echo sprintf('%01.4f', microtime(true) - $start_time);

What’s going on? First I include Zend_Cache. Next I declare two arrays with configuration values needed to use Zend_Cache. The frontendOptions array is setting a cache lifetime of 10 seconds. Meaning, after a cache file is created, it will only live for 10 seconds. After that, the cache file will be recreated. In the backendOptions array, I set the folder where I want my cache files to be saved to. I’m using a folder called ‘tmp’ that I created in the root of my site. Make sure to create that folder or your code may not work.

Next I create my $cache variable telling Zend_Cache that I want to use ‘Core’ as the frontend and ‘File’ as the backend. $id is just any name you want to give to this particular cached value. If you wanted to cache two different variables, you would want to use two different id’s for each to not overwrite one-another. $start_time is going to track when my code started running so I can check how long it took to execute my loop at the end.

This is where it gets fun yet so simple. The if statement checks the following

  • $cache->load($id) will check to see if a valid cache file exists for that $id and return it.
  • $data is set to the return of $cache->load (whether there is or isn’t anything there)
  • Finally the if checks to see if there is NO data in $data and if so, processes the loop else echos ‘Running from Cache’

If the if statement determines there is NO data in cache, it will continue with the code to do a loop. The for statement will loop 1,000,000 – one million times, and append each number to the variable $data. After it is done running a million times, it saves the variable $data into cache using the $id declared in the $cache->load() call. When it’s done the code spits out the time it took to execute this code. In my server it’s usually around 0.4 seconds:

Without Cache
in seconds

Now if it run my code again, hitting refresh, the code will go to the if() and find there is a cache file for that $id and load it – so it will not run the for loop. In this scenario, the page usually only takes about .02 to .03 seconds to execute:

Using Cache
in seconds

That’s a nice improvement. For the next 10 seconds (while the cache file is valid since we set its lifetime to 10 seconds) the page will only take 0.03 seconds to run instead of 0.4 seconds. That’s over 15 times (94%) faster! “Rudimentary, this is a difference of serving 149 requests per minute versus 2307 requests per minute.” You can always look inside your tmp folder and see if files are being created there to make sure things are really working. If you delete them, next time you execute the page it should recreate the cache files.

Example 2 – Caching a Database RecordSet

This example will be extremely similar to the previous one. In both, we are setting a variable to hold a value, then we save that variable in cache. Each time we run the page we check for the cache file, if it exists we use it, else we query the database, get the recordset, and store it in cache.

Setting up the Database

In this example we will be using the same table ‘users’ that I used in some of my previous posts. Below is the create statement to create the table ‘users’ in the database ‘test’:

DROP TABLE IF EXISTS `test`.`users`;
CREATE TABLE  `test`.`users` (
  `users_id` int(10) NOT NULL AUTO_INCREMENT,
  `first_name` varchar(100) NOT NULL,
  `last_name` varchar(100) NOT NULL,
  PRIMARY KEY (`users_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Once you create the table insert the following data:

INSERT INTO users
VALUES (null, 'Joey', 'Rivera'), (null, 'John', 'Doe'), (null, 'Joey', 'Tester'),
(null, 'Joey', 'Test'), (null, 'Billy', 'Bob');

Ok, now the code. The objective will be to grab all the information for all the users in the table from the database. Once we have it, store it in cache until that table is updated and we need to query it again. We are going to use a null value for lifetime so that the cache file never expires and we are then going to manually – in code, delete the cache file when we know the table has been changed.

This is what the code looks like:

<?php
include 'library/Zend/Cache.php';

$frontendOptions = array(
   'lifetime' => null,
   'automatic_serialization' => true
);

$backendOptions = array(
    'cache_dir' => 'tmp/'
);

$cache = Zend_Cache::factory('Core', 'File', $frontendOptions, $backendOptions);
$id = 'rs';

$start_time = microtime(true);

if(!($data = $cache->load($id)))
{
    echo "Not found in Cache<br />";

    mysql_connect('localhost', 'user', 'password');
    mysql_select_db('test');
    $query = 'select * from users';
    $rs = mysql_query($query);

    $data = array();
    while($row = mysql_fetch_assoc($rs))
    {
    	$data[] = $row;
    }

    $cache->save($data);
}
else
{
	echo "Running from Cache<br />";
}

//echo '<pre>';
//print_r($data);
//echo '</pre>';
echo sprintf('%01.4f', microtime(true) - $start_time);

This code is very similar to the first example. The differences are I changed the frontendOptions lifetime value from 10 seconds to null – so the cache file never expires. Then I changed the $id value to ‘rs’ so it doesn’t overwrite the example one cache file. Now instead of looping in the if statement, I connect to the database, query the users table, and create an array of all the rows returns. Then I save it in cache and echo out the time it took to execute. The next time, the code will find the cache file for this $id and go to the else statement and then echo the time.

Time taken without cache (querying database):

in seconds
in seconds

Time taken with cache (not querying the database):

in seconds
in seconds

As you can see, big improvement again – 33 times (97%) faster. “This is a difference of serving 4,477 requests per minute versus 150,000 requests per minute!” Feel free to uncomment the echo/print_r to see the data from the array. You can then update one of the users name in the database and run this page again. Notice you don’t see the new change. This is because we told the cache file to never expire so no matter what changes you make to your users in the users table, this page will continue to load the data from the cache file.

Clearing the Cache File

Depending on your needs, you may want the cache file to expire every 5 minutes, 2 hours, each day, or never. Even if you set a time interval for the cache file to expire, you may at some point find yourself needing to clear the cache early so this will show you how.

Clearing the cache is as simple as calling $cache->remove($id).  We need to add some code to delete a cache file when the users table is updated.

<?php
include 'library/Zend/Cache.php';

$frontendOptions = array(
   'lifetime' => null,
   'automatic_serialization' => true
);

$backendOptions = array(
    'cache_dir' => 'tmp/'
);

$cache = Zend_Cache::factory('Core', 'File', $frontendOptions, $backendOptions);
$id = 'rs';

if(isset($_GET['form_submit']) && $_GET['form_submit'] == 'clear')
{
	$cache->remove($id);
}

$start_time = microtime(true);

if(!($data = $cache->load($id)))
{
    echo "Not found in Cache<br />";

    mysql_connect('localhost', 'user', 'password');
    mysql_select_db('test');
    $query = 'select * from users';
    $rs = mysql_query($query);

    $data = array();
    while($row = mysql_fetch_assoc($rs))
    {
    	$data[] = $row;
    }

    $cache->save($data);
}
else
{
	echo "Running from Cache<br />";
}

echo sprintf('%01.4f', microtime(true) - $start_time);
?>
<form method="get">
	<input name="form_submit" type="submit" value="reload">
	<input name="form_submit" type="submit" value="clear">
</form>

The only difference between this code and the one above is I added a form at the end of the page. This form has two buttons, one we will use to reload the page without clearing the cache and the other button we will use to clear the cache after the page is submitted. If you look at the php code, you’ll notice a new if statement.

if(isset($_GET['form_submit']) && $_GET['form_submit'] == 'clear')
{
	$cache->remove($id);
}

This code checks to see if you selected the ‘clear’ button. If so, it calls the remove method in $cache to clear the cached file for $id. Right above this we are still initializing $cache the same way we use it when we want to cache a variable and we are still using the same $id ‘rs’.

Example 3 – Caching a Page

This is probably the easiest of them all because you don’t need to specify an id. The page is cached based on the url as the id. All you need to do it call $cache-start() after initializing the cache variable using ‘Page’ as the frontend. Everything else happens behind the scene. You don’t even have to call the save() method since anything outputted by the page will be cached. Here is the code, modified from example 1:

<?php
include 'library/Zend/Cache.php';

$frontendOptions = array(
   'lifetime' => 10,
   'automatic_serialization' => true
);

$backendOptions = array(
    'cache_dir' => 'tmp/'
);

$cache = Zend_Cache::factory('Page', 'File', $frontendOptions, $backendOptions);
$cache->start();

$start_time = microtime(true);

echo "Not found in Cache<br />";

$data = '';
for ($i = 0; $i < 1000000; $i++)
{
    $data = $data . $i;
}

echo sprintf('%01.4f', microtime(true) - $start_time);

The few differences here are using ‘Page’ as the frontend instead of ‘Core’. Notice that I removed the if statement since we don’t have to check for a cache file, simply calling $cache-start() will do that for us. And finally I don’t save a variable to save since as mentioned above, $cache-start() takes care of this as well.

There should be a slight delay loading this page the first time since it’s not cached yet. If you reload the page multiple times, you’ll notice the value in the timer doesn’t change even though the page loads much quicker. This is because all the output sent to the browser is cached, including the timer. So in this scenario, the timer is pretty useless since the value will be correct only when the code runs outside of the cache every 10 seconds (the lifetime is set to 10 seconds instead of null for this example). Every 10 seconds of reloading this page, you should see the timer change value.

Additional Thoughts

Cache is a powerful tool. You can save resources and time which could also mean saving money (with very little extra code!). You can have a very dynamic site which requires lots of processing and heavily relies on a database queries, creating bottlenecks that could be easily alleviated by caching instead of buying a lot more expensive hardware.

There is a slight overhead in using cache though. This is because your system has to read and write to files where before it wouldn’t have had to. So make sure you are using cache in a way that makes sense. Don’t use it for the fun of using it, make sure you use it to solve a problem or make a process more efficient.

As seen from these three examples, there are many ways to cache, so you can be creative. Sometimes caching an array makes sense, sometimes caching a whole page doesn’t. You just need to think about what the need is and how to best address it.

Also know there are many other forms of caching that aren’t specific to php. Caching can happen at the client level just like it can happen at server level, not just at the code.

Feel free to leave any questions, comments, thoughts on this topic and thanks for reading.

*edit: Updated my math on improvement times plus added requests example from Wiseguy.

35 thoughts on “Caching using PHP/Zend_Cache and MySQL”

  1. Thank you. HTTP caching has great benefits and should also be used. The way I see it, the server is the first line of defense and what I would tackle first. Unlike a browser where I can’t control every environment, I can control what happens on the server and how the server reacts to requests.

    More info on client side caching for those interested:
    http://code.google.com/speed/page-speed/docs/caching.html#LeverageBrowserCaching
    http://betterexplained.com/articles/how-to-optimize-your-site-with-http-caching/

  2. Great post introducing and explaining caching!

    However, your math is wrong, and it doesn’t do you any justice. :-p You lose the emphasis of how truly drastic the improvement is that caching can make. An improvement of 13% doesn’t really sway me, and 33% is only decent. Thus, I am happy to report that these numbers are incorrect.

    Example 1:
    “the page will only take 0.03 seconds to run instead of 0.4 seconds. That’s over a 13% improvement.”

    Not so! Using the specific figures of 0.4012s and 0.0260s, that is over 15x (i.e., almost 94%) faster! Rudimentarily, this is a difference of serving 149 requests per minute versus 2307 requests per minute!

    Example 2:
    “big improvement again – 33% or so?”

    Same thing here as in Example 1. 0.0134s to 0.0004s is over 33x (i.e., over 97%) faster! This is a difference of serving 4,477 requests per minute versus 150,000 requests per minute!

    Caching is obviously a good thing, but you should point out that’s it’s a REALLY good thing. 🙂

  3. Joey, thank you, this tutorial is great for getting started with caching!

    I just have one suggestion:
    When you invoke the save() method of Zend_Cache_Core, do you think for the sake of clarity it would be better to explicitly define the ID of the data you’re caching?

    I personally think this might make more sense to someone just starting out, rather than using the pretty much undocumented behavior of Zend_Cache to use the most recently used ID when it’s omitted from the function call.

  4. Brian, first let me paste the comments from the save method in Zend_Cache_Core.php so others understand what you are talking about. The save method takes in the following parameters:

    @param mixed $data:
    Data to put in cache (can be another type than string if automatic_serialization is on)

    @param string $id:
    Cache id (if not set, the last cache id will be used)

    @param array $tags:
    Cache tags

    @param int $specificLifetime:
    If != false, set a specific lifetime for this cache record (null => infinite lifetime)

    @param int $priority:
    integer between 0 (very low priority) and 10 (maximum priority) used by some particular backends

    I think it’s up to personal preference. In my case, I don’t use it because I think very sequentially and know that save is going to save the data for the id I just tried to load. I would only use it if I wanted to save the data to a cache file that wasn’t implied. To be honest, using the id in the load and in the save methods – for me, feels a bit redundant.

  5. Its great article and to know more about the Zend_Cache .
    You have covered every portions , so a newbie can learn more quickly without knowing anything about Zendframework .
    Good luck and Have a great day .

  6. Brian Powers, I was just going to note this 🙂 Indeed, I think adding cache unique identifier is absolutely required, otherwise you will someday have a very bad day of debugging… It’s even more strange that official Zend documentation for Zend_Cache also ommits identifier when saving.
    Joey, thanks anyway for useful tutorial.

  7. Hi Joey,

    nice article. I just need to add one point:
    You are using “File” cache for demonstration. This is great but is not useful in big productive environments. And a server that uses filecache has an IO overhead which is never good.

    Using a RAM cache like Memcache that can be easily used on multiple servers would be he way better option if you plan to create a high traffic site.

  8. ag, interestingly enough, I just tried my example and it’s not working for me. I don’t know what’s changed on my environment from when I wrote this post to today but here is what I tried and worked for me. I set all the default_options for Page to true and one of them fixed it so my example now works again. Here is the doc:

    http://framework.zend.com/manual/en/zend.cache.frontends.html#zend.cache.frontends.page

    Here is what I did:

    $frontendOptions = array(
    ‘lifetime’ => 10,
    ‘automatic_serialization’ => true,
    ‘default_options’ => array(
    ‘cache_with_get_variables’ => true,
    ‘cache_with_post_variables’ => true,
    ‘cache_with_session_variables’ => true,
    ‘cache_with_files_variables’ => true,
    ‘cache_with_cookie_variables’ => true,
    ‘make_id_with_get_variables’ => true,
    ‘make_id_with_post_variables’ => true,
    ‘make_id_with_session_variables’ => true,
    ‘make_id_with_files_variables’ => true,
    ‘make_id_with_cookie_variables’ => true
    )
    );

    I probably don’t need all that but at the moment I don’t have time to test which fixed my problem or why I’m having this problem when it worked before. Let me know if this fixes your problem.

  9. Great article, thanks!
    For use in the Zend Framework, where would I put the code to cache the entire page? In the view or controller?
    thanks

  10. Eric,

    If your plan is to cache the entire page I would do it as early on as possible, maybe even in the bootstrap after you initialize Zend_Cache. At this point, Zend_Cache would already know everything it needs to know to check if the page is cached, which is the url.

    Depending on your needs, there may not be any benefit of doing this in the controller nor in the view since it’s just more processing/time the server has to spend when the page might already be cached.

  11. Hi,

    Zend_Paginator and cache. How can I read the datas that I have sent to cache

    I register the data to cache and I see the foliowing :
    zend_cache—Zend_Paginator_1_42242d5fa3c4e4b7758810c276163e8a

    but I can’t read.

    $request = $this->getRequest();
    $q = new Model();
    $paginator = Zend_Paginator::factory($q->fetchAll());
    $paginator->setCurrentPageNumber($request->getParam(‘p’));
    $paginator->setItemCountPerPage(40);
    $this->view->q = $paginator;

    $fO = array(‘lifetime’ => 3600, ‘automatic_serialization’ => true);
    $bO = array(‘cache_dir’=> APPLICATION_PATH . ‘/cache/’);
    $cache = Zend_cache::factory(‘Core’, ‘File’, $fO, $bO);
    Zend_Paginator::setCache($cache);

  12. I think I may create a quick blog post today about using Zend_Paginator and caching for reference. The only thing I notice in your code is

    Zend_Paginator::factory($q->fetchAll());

    Have you tried adding ->toArray() at the end of that to store the actual array and not recordset.

    Zend_Paginator::factory($q->fetchAll()->toArray());

    I haven’t tried the Zend_Paginator::setCache() options before, I’ll try it out later today and see how it goes.

  13. Hi Joey, I’m still confuse with Caching a Page.

    Where I put this code?
    In Controller?
    If I already used cache core a result from database, are we need a Caching a Page?

  14. Teddy,

    One way to use page cache would be to set it up in your bootstrap and in the controller you can call cache->start() to load that page request from cache. That would work just fine if you only want to cache specific pages. If you want to cache everything (and you may not) you can just add cache->start() in your bootstrap after you initialize it. It really just depends on what you are trying to do.

    If you are already using page cache, you probably won’t want to use core as well. But if you are caching different parts of your application, you could use page for some and core for others. In your cache, if you already have your recordset cached, you probably won’t need page cache as well unless you have a lot of other heavy lifting code that is going on. But I wouldn’t use both for the same page. Hope this helps.

  15. Hi

    I just found this blog and I would really appreciate your help. I have a problem with caching the entire page, I set all parameters as indicated and when I run application locally cache files are created and also used, so on my machine is working. But, when I upload it on remote server which is hosting the site which NEEDS to be cached, files are created but never used! I see that every time request to the db has been made and created cache files are not used at all! And I don’t have any idea why???

    Heeeeelp! I really don’t know what to do and it’s driving me crazy 🙁

    1. Milica,

      I had this exact problem with my blog a while back and I don’t remember what I ended up doing about it. In the next couple days I’ll try to reproduce this and see if it’s still happening to me to some up with some solution.

  16. Thanks a lot! I would really appreciate it 🙂
    I have deadline till Monday to solve this issue and I find zend page caching the quickest solution to my problem and I don’t have time to implement other methods 🙁

    But it’s really strange and I can’t come up with anything why is it happening….

    Just for you to know, I am initializing caching inside Bootstrap.php on the top before any other _init method and I’m starting cache ($cache->start()) from there and I listed all default_options there as you said (because even locally it didn’t want to work if I don’t put it there)…

    Thanks again

  17. Milica,

    Try the following example on your server.

    http://www.joeyrivera.com/projects/cache_test/

    You can see all the code in that link. basically it shows a page and caches it for 10 seconds. If you reload and it’s cached, you should see a ‘page is cached’ message. See if you can copy/paste this code to your server and if you get the same results. Also, I would recommend turning on all error/warnings in case something is happening behind the scenes.

  18. Hi all experts,
    I’m new to Zend, I’ve many questions and confusions in my mind to get started with caching.
    It would be grateful if someone answer me to clear my mind.
    1) In What circumstances we need to use caching?
    2) How to use it for data that is frequently updated?
    3) How would we use it in select query which has WHERE clauses and complex sub queries in it like JOINS etc.
    4) For instance a user updates his profile, then other will see it if from cache then how would system fetch it from DB if the record was updated.
    Hope, I could mention my questions.

    Thanks for your response.
    shahzad

Leave a Reply

Your email address will not be published. Required fields are marked *