URL redirection with SmartCrawl (internal)

Internal note for VM staff. This page used to be a copy of WPMU DEV’s own SmartCrawl documentation, which is both a copyright problem and a maintenance one, because their docs change and ours did not. It has been replaced with the part that is actually ours: how redirects behave on VM sites, what bites, and how to set them without the interface.

Where redirects live

In the interface: SmartCrawl, then Advanced Tools, then URL Redirection. Add the source, add the destination, pick the type.

Underneath, they are rows in the wp_smartcrawl_redirects table (site prefix varies; see the client’s access note). Four columns matter when writing them programmatically:

• source stores the path with a trailing slash.
• path stores it without. Both need to be right or matching silently fails.
• destination is JSON-encoded and should be relative, not absolute.
• rules takes [] when there are no conditions.

⚠️ The four things that waste an afternoon

1. Redirects default to 302, not 301. A 302 is temporary and does not pass ranking. Always pass 301 explicitly when creating them in code, and check the type when creating them in the interface. This is the single most common mistake on this plugin.
2. Never test a redirect with a query string on the end. A URL like /old-page/?x=1 bypasses SmartCrawl’s matching entirely and returns the original page, so you conclude the redirect is broken when it is fine. Test the bare URL, in a private window, with caching in mind.
3. The option name is wds-advanced, with a hyphen. Not an underscore. Guessing the underscore version returns nothing and looks like the feature is off.
4. Flushing. After bulk changes, run wp rewrite flush. If the sitemap is involved, SmartCrawl’s sitemap cache also needs invalidating by hand: SmartCrawlSitemapsCache::get()->invalidate(). It does not clear itself, and a stale sitemap listing redirected or noindexed URLs tells Google two contradictory things.

House rules for redirects on client sites

• 301 unless there is a reason. Temporary means temporary.
• Redirect to the closest equivalent page, not the home page. A mass redirect to the home page is treated as a soft 404 and passes nothing.
• Never chain. If A already points to B and B is being retired, repoint A to C as well. Chains lose value and eventually loop.
• Record what you did in the client’s project note. Six months on, an unexplained redirect is indistinguishable from a mistake.
• Retiring a page: redirect first, then trash, then flush the sitemap cache, then verify. In that order.

Verifying

Check the status code, not the browser’s behaviour, because the browser will happily follow anything:

curl -s -o /dev/null -w "%{http_code} %{redirect_url}n" https://example.com/old-page/

You want 301 and the expected destination. If you get 200, the redirect is not matching. If you get 302, the type is wrong.

Related

• What not to touch, and when to email us instead