Fetch_feed Not Working With URLs Containing & Symbol

I recently came across a strange problem with the fetch_feed function in WordPress not working with URLs containing the & symbol. The URL was being passed to fetch_feed via a shortcode parameter, but fetch_feed couldn’t find the feed.

Background

The URL I was working with was the feed for the Queensland Police Service Facebook page:

http://www.facebook.com/feeds/page.php?id=339665603253&format=rss20

It was being passed to a function that called fetch_feed on it, via a shortcode:

[facebook_feed url="http://www.facebook.com/feeds/page.php?id=339665603253&format=rss20" title="Queensland Police Facebook" number="4"]

The shortcode function was doing something like this (extra code stripped):

$url = esc_url ( $attribute_values[ 'url' ] ); 
$feed = fetch_feed( $url );	

This worked flawlessly from the sidebar (I had enabled shortcodes in the sidebar), but it was failing when the shortcode was called via a post.

Debugging The Problem

I had trouble debugging the problem because of the way the & & and & characters are handled by WordPress and / or the browser.

See what I mean. If you view the generated source of this page you will find that all three of those characters appear as &. If you view the raw source, they appear as & & and &. In the editor, I actually typed & & and & respectively. Confusing!

Anyway, I did a var_dump on the $feed variable, so I could see the WP_Error object and it gave me:

object(WP_Error)#4 (2) { ["errors"]=> array(1) { ["simplepie-error"]=> array(1) { [0]=> string(101) "A feed could not be found at http://www.facebook.com/feeds/page.php?id=339665603253&format=rss20" } } ["error_data"]=> array(0) { } }

The problem is the URL in the error is correct and works when I paste it into the browser. I had a look at the source and found it had & instead of &. That doesn’t work (come on Facebook, it should!). But then all of the ampersand characters were showing up as &, so I didn’t trust it.

To find out what was really in the URL, I added spaces between the characters, so that the browser wouldn’t be able to render & and &:

for ( $i = 0; $i < strlen( $url ); $i ++ ) {
	echo substr( $url, $i, 1) . ' ';
}

That gave me the following (where we can now see the & # 0 3 8 ; ):

h t t p : / / w w w . f a c e b o o k . c o m / f e e d s / p a g e . p h p ? i d = 3 3 9 6 6 5 6 0 3 2 5 3 & # 0 3 8 ; f o r m a t = r s s 2 0

What I didn’t realise at the time was that I made a mistake in looking at the generated source (View Selection Source in Firefox), rather than the raw source. Had I looked at the raw source, I would have seen the &#038; straight away – not that it makes a difference, the URL doesn’t work with that in it either.

I was running the url parameter through the esc_url function for security purposes. I tried moving to the urlencode / urldecode combination, but that just changed the &#038; to &amp; and it still didn’t work. The problem was that the URL need it to remain as & for fetch_feed to work.

Solution

At first I thought WordPress was at fault and I disabled wptexturize and wpautop from running on the_content, but this didn’t make a difference. It could be that WordPress is doing something else that I’ve missed (after all it works in the widget, just not the post), but I couldn’t see where.

I then tried using str_replace to remove the #038;, leaving just the & in place and it worked! Just as I was settling on this as the solution, I found an undocumented function in the WordPress core, wp_specialchars_decode, which worked without needing the str_replace.

So my final solution was:

$url = esc_url ( $attribute_values[ 'url' ] ); 
$feed = fetch_feed( wp_specialchars_decode($url) );	

I wasted about half a day on this, when I just need to know about wp_specialchars_decode. It doesn’t appear in any of the numerous fetch_feed tutorials out there, but hopefully people with a similar problem will find this.

4 responses on “Fetch_feed Not Working With URLs Containing & Symbol

  1. @metacowboy

    thanks a lot Stephan you saved me half a day finding the bug . Was shortly to get into the core of feed.php Works now like a sharm with the Facebook feed urls

  2. Adnauseum

    I wish your code above would fix our template. No matter what we try, in the URL in a slider such as /client/cart.php?a=pid2&

    The only thing found in the /themes/edge/library is a file loop-pagination.php which has:

    /* Remove ‘page/1′ from the entire output since it’s not needed. */
    //$page_links = str_replace( array( ‘&paged=1\”, ‘/page/1\”, ‘/page/1/\” ), ‘\”, $page_links );

    I put the two // in front but no help there.

  3. BjornW

    Stephan, you can also use esc_url without the wp_specialchars_decode function by setting the context parameter of the esc_url function to something else than the default ‘display’ value. This prevent the esc_url function from replacing the ampersands and single quotes. You may also just use esc_url_raw() which is a wrapper function for esc_url($url, $protocols=null, ‘db’). See also: http://core.trac.wordpress.org/browser/tags/3.2.1/wp-includes/formatting.php#L2259

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Anti-Spam Quiz: