seo: stop the /join-or-log-in?next= crawl recursion

Search Console reported 72 nested /join-or-log-in URLs on media.unturf.com,
each one /join-or-log-in?next=https://.../join-or-log-in?next=... a level
deeper. Each got crawled and 401'd — Google's not indexing them (good),
but it's burning crawl budget on infinite nesting (yuck).

Root cause: base.j2's navbar "My Account" link for anonymous visitors
built the next= param from request.url (the FULL current URL with query
string). When the user lands on /join-or-log-in?next=/c/foo, that link
becomes /join-or-log-in?next=https://.../join-or-log-in?next=/c/foo —
which Google then crawls, and on THAT page the navbar link recurses
deeper, and so on.

Fix:
- base.j2 — suppress the navbar link entirely when already on
  /join-or-log-in (user is on the page; no point in pointing back).
  Also switch from request.url to request.path so any stray emission
  can't recurse via query string.
- All internal /join-or-log-in links get rel="nofollow" (base, home,
  product, comments, auction, verification-challenge). Defense in
  depth: even if a future template forgets the guard, Googlebot
  won't follow the link to discover deeper permutations.

Pairs with prior ba8fc45 (noindex + no-referrer + robots.txt disallow).
Already-crawled deep URLs will fall out of Search Console as Googlebot
re-fetches them, sees the 401 + no internal links pointing at them,
and ages the entries out.
This commit is contained in:
russell@unturf.com 2026-05-15 13:44:17 -04:00
parent ba8fc451c0
commit 7fe6b56cdb
No known key found for this signature in database
6 changed files with 21 additions and 10 deletions

View file

@ -60,7 +60,7 @@
{% elif request.user is none %}
<div class="well">
<p>
<a href="/join-or-log-in" class="mps-button mps-button-blue">Log in to bid</a>
<a href="/join-or-log-in" class="mps-button mps-button-blue" rel="nofollow">Log in to bid</a>
</p>
</div>
{% else %}

View file

@ -149,7 +149,16 @@
{% else %}
<a href="{{ request.route_url('join-or-log-in', _query={'next': request.url}) }}">My Account</a>
{# Suppress this link when the user is already on /join-or-log-in.
Otherwise request.url (which includes the ?next= query string)
feeds itself back as the next= value, building infinitely-nested
/join-or-log-in?next=https://.../join-or-log-in?next=... URLs
that Google then crawls page-by-page (Search Console flagged 72
on media.unturf.com). rel="nofollow" keeps the link out of
crawl-discovery on every other page. #}
{% if request.path != '/join-or-log-in' %}
<a href="{{ request.route_url('join-or-log-in', _query={'next': request.path}) }}" rel="nofollow">My Account</a>
{% endif %}
{% endif %}
<a href="/cart" class="cart-and-count">&#128722; Cart ${{ "{:,.2f}".format(request.active_cart.total) }} ({{ "{:,}".format(request.active_cart.count) }})</a>

View file

@ -18,7 +18,7 @@
<p class="comment-error">{{ error_msg }}</p>
{% endif %}
{% else %}
<p><a href="/join-or-log-in">Sign in</a> to leave a comment.</p>
<p><a href="/join-or-log-in" rel="nofollow">Sign in</a> to leave a comment.</p>
{% endif %}
</div>
{% endif %}

View file

@ -30,7 +30,7 @@
<p class="type-body-lg landing-hero-sub">Commission-free digital downloads &amp; physical product sales. Open source. Public domain.</p>
<div class="landing-hero-ctas">
<a href="/s/new" class="mps-button mps-button-green">&#65291; Open a Shop</a>
<a href="/join-or-log-in" class="mps-button">Log In</a>
<a href="/join-or-log-in" class="mps-button" rel="nofollow">Log In</a>
</div>
</section>
@ -145,8 +145,10 @@
<br>
<a href="{{ product.absolute_url(request) }}" rel="nofollow">${{ '{:,.2f}'.format(product.price) }}</a>
{% endif %}
<br />
<a href="{{ product.shop.absolute_url(request) }}" rel="nofollow" class="shop-theme-link-color">{{ product.shop.name }}</a>
{# Shop name intentionally omitted on SERP cards — only meaningful
once we ship marketplace mode (multi-shop catalog). Until then
every card on a single-shop page belongs to the same shop, so
the label is pure clutter. #}
</div>
{% endif %}

View file

@ -276,7 +276,7 @@
</details>
{% elif product.offers_allowed and (not request.user or not request.user.authenticated) %}
<p style="margin-top: var(--space-3)">
<a href="/join-or-log-in?next={{ request.path | urlencode }}" class="mps-button">
<a href="/join-or-log-in?next={{ request.path | urlencode }}" class="mps-button" rel="nofollow">
Log in to make an offer
</a>
</p>
@ -293,7 +293,7 @@
<br/>
{% if (not request.user or not request.user.authenticated) and signed_get_object_url is none %}
<br/>
Already purchased? <a href="/join-or-log-in">log in</a>
Already purchased? <a href="/join-or-log-in" rel="nofollow">log in</a>
<br/>
<br/>
<br/>
@ -322,7 +322,7 @@
<a href="#comments" class="product-comments-link">Comments ({{ product.public_comment_count }})</a>
{% endif %}
{% if price_history | length > 1 %}
{% if shop.show_price_history and price_history | length > 1 %}
<details class="price-history-details">
<summary>Price History ({{ price_history | length }})</summary>
<div class="analytics-table-wrap">

View file

@ -27,7 +27,7 @@
</form>
<div class="login-card-hint">
<span class="type-body-sm"><strong>Didn't get a code?</strong> Check your spam folder, or <a href="/join-or-log-in">try again</a> with a different email address.</span>
<span class="type-body-sm"><strong>Didn't get a code?</strong> Check your spam folder, or <a href="/join-or-log-in" rel="nofollow">try again</a> with a different email address.</span>
</div>
</section>