The transients feature of WordPress is a very useful tool for local data cache when pulling from a remote data source (web service/API).
A typical workflow goes something like this:
- Get remote data.
- Store it locally as transient data with a timeout.
- Once the data times out, get remote data again.
That’s the basics and will work well most of the time. However there are a few considerations to keep in mind when using transients to store this data.
- When your transient expires, it is deleted from the WordPress database.
- You cannot trust your remote data source to be accessible 100% of the time.
To account for the situations where a transient has expired and you can’t reach your remote data source, you will want to store the latest copy of your remote data in a more permanent location (probably as an option or post meta, depending on the nature of the data). With this copy of the data still available, you can insulate yourself from situations where your remote data source is unavailable.
Here is a better workflow that does error handling in a graceful and localized way:
- Get remote data.
- Store it locally as transient data with a timeout.
- Also store it locally as non-transient data (as an option or post meta).
- Once the data times out, get remote data again.
- If remote data request fails, fall back on the non-transient copy of the data and try to get the remote data again next time.
You may want to implement some throttling as well (re-check every minute rather than every request), if so – you can use a transient to easily implement the throttle duration.
Hopefully this is useful to other WordPress devs out there. Other (better?) suggestions on how to handle this? Share them in the comments.
Yep, fallback is exactly what a lot of code that makes use of transient lacks.
The one issue with improved workflow that it is messy. Transients are tidy – they self-destruct. Options aren’t – you need to keep track so that they don’t stuck in database.
It would be great if transients had two timers – period after which they expire and period after which they are deleted. I’ve been trying to write something like that for background feed fetches, but it got complex and I decided to leave it for some time later.
In this particular situation I wouldn’t use the transient API. I’d store the data as a persistent option that gets updated periodically with a WP cron schedule. If the external request fails you still have your option stored, unlike a transient which will have expired.
Additionally, if the external request fails you can add in an extra single cron event that fires within a short period of time rather than waiting for the next scheduled update.
Hey Alex,
Developers shouldn’t expect that transients are stored in the database, as the good object caching layers out there handle these on their own.
Because of that, it probably isn’t a good idea to store data separately in an option or postmeta, because then you em>are dealing with the database when you might not otherwise be doing. (Granted, there are object caching mechanisms for that too.)
What I would do is actually use two transients, one with an much longer timeout than the other. Always try to grab the shorter one if possible. If you need to remote fetch the shorter one, and it is successful, then also reset the content of the longer one. If it fails, then use the longer one.
I’ve also seen this strategy used in the WP_Object_Cache API, as those can take timeouts too.
It’d probably be cool if this can be leveraged in core in some way. Suggestions welcome.
Nacin
Where you store the fallback data is a separate issue IMO – you should definitely use a storage options that matches the needs of your site. These are intended to be easily accessible examples (hence using well understood constructs).
[…] This post was mentioned on Twitter by Rarst, Linda Schenk, blognews, Emil Uzelac, WordPress Geek and others. WordPress Geek said: Alex King: Using WordPress Transients with External Data http://goo.gl/fb/O3iWp […]
I don’t see the use of this idea.
If you’re gonna store the data in option/content-meta, why store it as transient?
It’s easier to just store it as traditional data and together store a expiration time. When it expires, you give priority to retrieving updated data, and if it fails you use the expired data until you can retrieve the updated one.
With your idea you’re just wasting space and CPU time for storing same data twice.
Also, if you’re using permanent caching, both option/content-meta and transient are stored in the same place, so performance is the same.
Transient means a chaching that can also be used when permanent traditional caching isn’t available (when permanent caching is used transient is saved there, otherwise it’s saved as a normal option), for resources that are very costly to retrieve and really need caching, but that also expires and requires updating.
Transient when permanent caching isn’t in use is the abstraction of a traditional option that’s deleted when expired. It’s meaning is to make everything simpler in the way we don’t need to keep testing for permanent caching to know where to store our temporary data and not needing to manually store in database when it will expire.
If data expired, it should be deleted because it’s useless, that’s the semantic of transient. If you don’t want it deleted upon expiration, use normal option with an extra option for the expiration data… transient isn’t for you after all.
Also, remember that when permanent caching is in use, you are not secured that cached data will remain saved even before it expires, so you can’t count at all that it’s gonna be there, so caching and transient isn’t for important data that can’t be lost and can’t be recreated anytime needed.