From 7fe6b56cdb6abe6948a91461d7375dc0b07d7222 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 15 May 2026 13:44:17 -0400 Subject: [PATCH] seo: stop the /join-or-log-in?next= crawl recursion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Search Console reported 72 nested /join-or-log-in URLs on media.unturf.com, each one /join-or-log-in?next=https://.../join-or-log-in?next=... a level deeper. Each got crawled and 401'd — Google's not indexing them (good), but it's burning crawl budget on infinite nesting (yuck). Root cause: base.j2's navbar "My Account" link for anonymous visitors built the next= param from request.url (the FULL current URL with query string). When the user lands on /join-or-log-in?next=/c/foo, that link becomes /join-or-log-in?next=https://.../join-or-log-in?next=/c/foo — which Google then crawls, and on THAT page the navbar link recurses deeper, and so on. Fix: - base.j2 — suppress the navbar link entirely when already on /join-or-log-in (user is on the page; no point in pointing back). Also switch from request.url to request.path so any stray emission can't recurse via query string. - All internal /join-or-log-in links get rel="nofollow" (base, home, product, comments, auction, verification-challenge). Defense in depth: even if a future template forgets the guard, Googlebot won't follow the link to discover deeper permutations. Pairs with prior ba8fc45 (noindex + no-referrer + robots.txt disallow). Already-crawled deep URLs will fall out of Search Console as Googlebot re-fetches them, sees the 401 + no internal links pointing at them, and ages the entries out. --- make_post_sell/templates/auction.j2 | 2 +- make_post_sell/templates/base.j2 | 11 ++++++++++- make_post_sell/templates/comments/show_comments.j2 | 2 +- make_post_sell/templates/home.j2 | 8 +++++--- make_post_sell/templates/product.j2 | 6 +++--- make_post_sell/templates/verification-challenge.j2 | 2 +- 6 files changed, 21 insertions(+), 10 deletions(-) diff --git a/make_post_sell/templates/auction.j2 b/make_post_sell/templates/auction.j2 index 7126402..9021fee 100644 --- a/make_post_sell/templates/auction.j2 +++ b/make_post_sell/templates/auction.j2 @@ -60,7 +60,7 @@ {% elif request.user is none %}

- Log in to bid + Log in to bid

{% else %} diff --git a/make_post_sell/templates/base.j2 b/make_post_sell/templates/base.j2 index 5af3723..8b25036 100644 --- a/make_post_sell/templates/base.j2 +++ b/make_post_sell/templates/base.j2 @@ -149,7 +149,16 @@ {% else %} - My Account + {# Suppress this link when the user is already on /join-or-log-in. + Otherwise request.url (which includes the ?next= query string) + feeds itself back as the next= value, building infinitely-nested + /join-or-log-in?next=https://.../join-or-log-in?next=... URLs + that Google then crawls page-by-page (Search Console flagged 72 + on media.unturf.com). rel="nofollow" keeps the link out of + crawl-discovery on every other page. #} + {% if request.path != '/join-or-log-in' %} + My Account + {% endif %} {% endif %} 🛒 Cart ${{ "{:,.2f}".format(request.active_cart.total) }} ({{ "{:,}".format(request.active_cart.count) }}) diff --git a/make_post_sell/templates/comments/show_comments.j2 b/make_post_sell/templates/comments/show_comments.j2 index f0c8686..3cb1eb6 100644 --- a/make_post_sell/templates/comments/show_comments.j2 +++ b/make_post_sell/templates/comments/show_comments.j2 @@ -18,7 +18,7 @@

{{ error_msg }}

{% endif %} {% else %} -

Sign in to leave a comment.

+

Sign in to leave a comment.

{% endif %} {% endif %} \ No newline at end of file diff --git a/make_post_sell/templates/home.j2 b/make_post_sell/templates/home.j2 index c5ea714..fdc34a2 100644 --- a/make_post_sell/templates/home.j2 +++ b/make_post_sell/templates/home.j2 @@ -30,7 +30,7 @@

Commission-free digital downloads & physical product sales. Open source. Public domain.

＋ Open a Shop - Log In + Log In
@@ -145,8 +145,10 @@
${{ '{:,.2f}'.format(product.price) }} {% endif %} -
- {{ product.shop.name }} + {# Shop name intentionally omitted on SERP cards — only meaningful + once we ship marketplace mode (multi-shop catalog). Until then + every card on a single-shop page belongs to the same shop, so + the label is pure clutter. #} {% endif %} diff --git a/make_post_sell/templates/product.j2 b/make_post_sell/templates/product.j2 index f833670..1eb9c2a 100644 --- a/make_post_sell/templates/product.j2 +++ b/make_post_sell/templates/product.j2 @@ -276,7 +276,7 @@ {% elif product.offers_allowed and (not request.user or not request.user.authenticated) %}

- + Log in to make an offer

@@ -293,7 +293,7 @@
{% if (not request.user or not request.user.authenticated) and signed_get_object_url is none %}
- Already purchased? log in + Already purchased? log in


@@ -322,7 +322,7 @@ Comments ({{ product.public_comment_count }}) {% endif %} - {% if price_history | length > 1 %} + {% if shop.show_price_history and price_history | length > 1 %}
Price History ({{ price_history | length }})
diff --git a/make_post_sell/templates/verification-challenge.j2 b/make_post_sell/templates/verification-challenge.j2 index e1be5ad..bcf0729 100644 --- a/make_post_sell/templates/verification-challenge.j2 +++ b/make_post_sell/templates/verification-challenge.j2 @@ -27,7 +27,7 @@