engineering/make_post_sell
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases 2 Packages Wiki Activity Actions

seo: mark /join-or-log-in as crawler-unfriendly + no Referer leakage

Browse source 
Google Search Console flagged six /join-or-log-in?next=... URLs as
"Duplicate without user-selected canonical" — every ?next= permutation
serves identical content, so Google groups them as duplicates and
indexes none. Auth pages have no business in the index anyway, and
the ?next= query string can carry private cart/product paths that
shouldn't end up in crawl logs or Referer headers.

Two-pronged fix:

  templates/join-or-log-in.j2
    - <meta name="robots" content="noindex,nofollow"> drops the page
      and its outbound links from the index entirely.
    - <meta name="referrer" content="no-referrer"> stops the URL —
      with its ?next= cart-uuid / product-uuid contents — from leaking
      to any third party when the user clicks an outbound link.
    - Explicit <title>Log in or join</title> via the head block so
      the default {{ request.domain }} title isn't shown either.

  views/misc.py — DEFAULT_ROBOTS_DOT_TXT
    - Disallow: /join-or-log-in so crawlers don't fetch the URL in
      the first place. Pages already in the index will drop out once
      Google re-crawls and sees the noindex header.

Operator shops can still override robots_dot_txt in pillar config;
they get the same default unless they explicitly opt out.
This commit is contained in:
russell@unturf.com 2026-05-15 13:29:32 -04:00
parent adf145a7e8
commit ba8fc451c0
No known key found for this signature in database
2 changed files with 13 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
make_post_sell/templates/join-or-log-in.j2
View file

@ -1,4 +1,16 @@
{% extends "base.j2" -%}
{%- block append_to_head_tag_section %}
<title>Log in or join</title>
{# Auth gateway — every ?next=... permutation serves identical content,
which Google Search Console flagged as "Duplicate without
user-selected canonical". noindex,nofollow keeps the auth page out
of the index entirely (it has no business being indexed); the
no-referrer policy stops the URL — which can contain private
cart/product paths in ?next= — from leaking via Referer when the
user clicks any outbound link. #}
<meta name="robots" content="noindex,nofollow" />
<meta name="referrer" content="no-referrer" />
{%- endblock append_to_head_tag_section %}
{% block content -%}
<section class="one-column">
<section class="login-card elevation-3 animate-fade-in-up">

2
make_post_sell/views/misc.py
View file

@ -16,7 +16,7 @@ from make_post_sell.lib.render import markdown_to_html
DEFAULT_ROBOTS_DOT_TXT = """
User-agent: *
Disallow:
Disallow: /join-or-log-in
"""