Python is worth using for SEO when you have a repeated job with a stable input and an output someone can test. Google Antigravity is an agentic development platform with an editor, terminal-oriented workflows, and agent coordination surfaces. Use it to build a very small command-line interface (CLI), not a giant crawler that touches production before you understand its failure modes.
The project in this guide reads a small URL fixture or sitemap sample and writes a page-signal report. It does not log into a site, crawl without limits, diagnose indexing, or change a live page.
Define the output contract before you write Python
Tell the agent what a correct output looks like. The contract is more valuable than a clever scraper.
requested_url,checked_at,status_observed,final_url_observed,title,h1,fetch_state,notes
https://example.com/guide/,2026-07-30T09:00:00Z,200,https://example.com/guide/,"Guide title","Guide heading",ok,""
https://example.com/missing/,2026-07-30T09:00:02Z,404,not_checked,"","",http_error,"review release or route"Rules for this first version:
- Read a local CSV with no more than 25 public URLs.
- Use a fixed timeout and a low request rate.
- Preserve requested URLs exactly in the output.
- Record
not_checkedrather than inventing a final URL, title, or H1. - Write a CSV and a short run summary.
- Exit with a useful error when the input is malformed.
Definition of done: pytest passes on fixtures for a normal page, a redirect, an error page, a timeout, and a malformed input row. The tool has not been pointed at your full site.
Build fixtures before the network layer
Create a folder that separates test data from generated reports:
seo-page-cli/
src/
tests/
fixtures/
urls-valid.csv
urls-malformed.csv
page-ok.html
page-no-h1.html
reports/
pyproject.toml
README.mdWhy fixtures first? A live URL can change while you debug. A fixture gives you a repeatable statement such as: "when the HTML contains one H1, the report has that exact H1." That lets you test parsing without blaming a network request.
Give Antigravity a narrow build request
Use Antigravity's agent workflow for a local project. Ask it to create the smallest implementation that satisfies the contract.
Create a local Python project named `seo-page-cli`.
Goal: read a local CSV of up to 25 public URLs and write a page-signal CSV.
Use the provided output schema and fixtures. Implement input validation, a
bounded HTTP client with a fixed timeout, HTML title/H1 extraction, explicit
fetch states, and a Markdown run summary. Add pytest tests for the supplied
fixtures and mocked HTTP responses for 200, redirect, 404, timeout, and
malformed input.
Do not run against a production sitemap, use credentials, add browser
automation, follow unlimited redirects, increase concurrency, modify websites,
or claim index status. Show a plan and file list before writing code.The plan should describe a parser, fetch function, report writer, and tests. If it offers a database, a dashboard, an LLM classifier, or a full-site crawl, decline those extras. They solve problems you have not earned yet.
Inspect the code through failure states
Run the test suite after the first implementation. A passing happy-path test is not enough.
Fixture or mock | Expected state | What it protects you from |
|---|---|---|
Valid input, 200 response |
| A report that omits basic signals |
Redirect response | Observed final URL with a bounded redirect policy | Treating every requested URL as final |
404 response |
| Inventing page signals after an error |
Timeout |
| A run that hangs forever |
Bad CSV row | Validation error with line number | Quietly losing URLs |
HTML without H1 |
| Turning an absent element into an indexing conclusion |
Ask the agent to explain each test in a sentence. If it cannot, the test may be testing an implementation detail instead of the report contract.
A good report distinguishes "the check failed" from "the page has a problem." Those are not the same thing.
Use a local sample before a sitemap
Now run the CLI against tests/fixtures/urls-valid.csv and inspect the generated report. Check row counts, source URLs, timestamps, columns, and failure-state wording. Then create a new local file with a small approved sample of public URLs from your own site.
python -m seo_page_cli check input/approved-sample.csv --output reports/page-signals.csv
pytestThe exact command may differ from your project. What matters is that it is documented in README.md, uses local input, and produces a file you can inspect.
If the sample report finds a missing title or H1, treat it as a lead. Open the rendered page in a normal browser and inspect the source or template before changing anything. A simple request can miss client-side rendering, consent behavior, bot handling, or another response path.
Add scope only when the first version is boring
After a few successful runs, you may add sitemap parsing, canonical extraction, or scheduled internal reports. Add one capability at a time, with fixtures and tests. Do not jump from a 25-URL local sample to an unbounded production crawl.
Use this expansion gate:
- The current output has a stable schema and a review owner.
- Tests cover the new behavior and failure state.
- Request limits and timeouts are documented.
- The site owner approves the target scope.
- The output still labels unobserved values clearly.
- The change does not write to a CMS, redirect config, robots file, or search-engine service.
Google's sitemap documentation is helpful when you later add parser support. It does not turn a sitemap into an index-status report. Keep crawlability and indexing questions separate from the page-signal CSV.
A practical prompt for a repair run
When a test fails, use a prompt that protects the contract:
`tests/test_report.py::test_timeout_row` is failing. Diagnose it using the
existing output contract and fixtures. Propose the smallest code change, explain
what report value will change, and add or adjust only the relevant test. Do not
rewrite the project, add dependencies without explaining them, or run the tool
against external URLs. Show the diff and test result.That prompt is much safer than "fix the crawler." It makes the test, the report, and the desired behavior part of the same conversation.
FAQ
Does this CLI tell me whether a page is indexed?
No. It records what the tool observed from a bounded request. Indexing requires separate evidence, such as appropriate Search Console investigation.
Should I build a crawler with Antigravity first?
No. Begin with fixtures and a small approved URL file. Crawl scope, rate limits, rendering, robots behavior, authentication, and data retention all need deliberate decisions.
Can the CLI submit URLs to Google or Bing?
Do not add that to this starter tool. Submission is an external write with its own approval path. Keep the first project read-only so that its reports can become trustworthy.
Author: Julian Mercer, 14-Year Technical SEO Practitioner at Auspia. Julian writes about testable technical SEO workflows and the evidence needed before teams touch production.
