Как анализировать трафик сайтов конкурентов с помощью Codex: Ahrefs, Semrush и DataForSEO

mkdir -p .agents/skills/competitor-traffic-research

---
name: competitor-traffic-research
description: Analyze a public competitor domain's estimated search traffic, top pages, ranked keywords, trends, and content gaps with Ahrefs, Semrush, or DataForSEO data. Use when a user asks to check competitor website traffic, competitor organic keywords, top pages, or organic search competitors. Set up any required provider connection, then produce a read-only, sourced report without inventing metrics.
---

# Competitor traffic research

## Purpose and boundary

Turn a public domain into a reviewable competitor-search report. This is research only. Do not edit a website, create a provider project, alter billing, change account settings, publish content, send email, or make any other external write action.

All third-party traffic values are estimates unless the user supplies first-party analytics for a domain they own. Never call an estimated value "actual traffic," "sessions," "revenue," or "conversions."

## Required input

Ask only for what is missing:

1. Competitor domain, subdomain, path, or exact URL. Normalize it and state which scope will be measured.
2. At least one available authorized provider: Ahrefs, Semrush, or DataForSEO.

Use `United States` and `English` when market and language are not supplied. State this default prominently in the final report. Optional inputs are the user's domain, additional competitors, device, date range, seed topic, and business goal.

## Provider connection and setup

Handle the provider setup so the user does not have to read API documentation or write request code.

1. Inspect installed provider skills and local integration scripts first. Prefer an existing authorized connection.
2. If no connection is ready, tell the user which provider connection is needed and guide them through its ordinary setup one step at a time. Use the provider's official documentation and configuration method; do not invent endpoints or settings.
3. Store provider settings in the location expected by the installed integration. Use `AHREFS_API_KEY`, `SEMRUSH_API_KEY`, `DATAFORSEO_LOGIN`, and `DATAFORSEO_PASSWORD` when a local script expects those names. Set DataForSEO's `DATAFORSEO_DEFAULT_LOCATION` and `DATAFORSEO_DEFAULT_LANGUAGE` to the selected market and language when the installed toolkit uses them.
4. Do not copy provider credentials into research artifacts, report tables, CSV output, or user-facing summaries. Keep connection setup out of the analysis report.
5. Use read-only reports/endpoints. Before a potentially billable request, state the provider, report or endpoint class, market, language, intended request count, and any known quota/credit uncertainty. Stop if the user declines.
6. On a 401, 403, quota, coverage, or provider error, record the provider as `unavailable` with the safe error category and a recovery suggestion. Do not retry repeatedly, switch providers silently, or invent a substitute metric.

## Provider query map

Choose the least complicated available route. Do not force all three providers into one run.

| Provider | First lookup | Extend the analysis with | Use it when |
| --- | --- | --- | --- |
| Ahrefs | Site Explorer domain overview or the installed Ahrefs connector's closest equivalent | Top pages, Organic keywords, Organic competitors, Content Gap | The connection exposes these reports and the user needs page-level competitor research |
| Semrush | Domain Overview or the installed Semrush connector's closest equivalent | Organic Research, Organic Competitors, Keyword Gap, historical position data | The connection exposes the target database and the user needs a second point of view or keyword-gap workflow |
| DataForSEO | `bulk_traffic_estimation` and `ranked_keywords` from the installed `dataforseo-toolkit` | `keywords_for_site`, `google_organic_serp`, `backlinks_summary`, `referring_domains` | A scriptable local workflow is available or a reproducible CSV is the main need |

For DataForSEO, use the installed toolkit before writing new HTTP code. Its normal sequence for a domain is:

```text
bulk_traffic_estimation -> ranked_keywords -> keywords_for_site
```

Run `google_organic_serp` only for a small, human-selected set of important keywords to validate intent and result-page format. Run backlink reports only when the user asks about referral or link opportunities. If an Ahrefs or Semrush connector uses different report names, use the closest documented read-only equivalent and record the exact report name.

## Data collection order

1. Create a timestamped folder under `competitor-research/` using a safe normalized domain name. Do not overwrite an existing run.
2. Create `research-scope.md` before any API call. Record target scope, market, language, date/time in UTC, available providers, user goal, and the meanings of requested metrics.
3. Inspect installed provider skills, local scripts, and official provider documentation before choosing a report/endpoint. Use only capabilities that are actually available to the authorized account. Do not guess an endpoint from memory.
4. Collect the smallest useful evidence set from each available provider:
   - domain-level estimated organic traffic or visibility and any available trend;
   - top organic pages with their leading keyword or traffic contribution when the provider supplies it;
   - ranked organic keywords with position, volume, and URL when available;
   - organic competitors or intersecting keywords when available;
   - paid-search or backlink signals only when the user asks for them, and label them separately.
5. If the user provides their own domain, run a content-gap comparison only when the provider supports it. Return only relevant keyword opportunities; do not treat every missing keyword as a content brief.
6. Normalize domains, country/database, language, device, date window, URL scope, and metric definitions before comparing providers. Keep each provider's original metric in a separate column. Never average or sum traffic estimates across providers.
7. Save concise normalized CSV files only: `competitor-pages.csv`, `competitor-keywords.csv`, and, when applicable, `content-gap.csv`. Omit unavailable fields rather than filling them with zero.

## Output schemas

Write a `field-dictionary.md` beside the report. Use these columns where the provider returns them; preserve blanks as blanks.

### `competitor-pages.csv`

`url, page_role, leading_keyword, estimated_organic_traffic, ranking_keywords, traffic_change, provider, report_or_endpoint, retrieved_at_utc, market, language, scope, metric_definition, notes`

### `competitor-keywords.csv`

`keyword, intent, position, previous_position, search_volume, estimated_traffic, ranking_url, keyword_group, provider, report_or_endpoint, retrieved_at_utc, market, language, scope, metric_definition, relevance_note`

### `content-gap.csv`

Create this file only when the user's domain is supplied and a provider supports a comparison. Use:

`keyword, intent, competitor_domains, competitor_urls, user_domain_status, search_volume, position_gap, recommended_action, existing_url_risk, provider, report_or_endpoint, retrieved_at_utc, market, language, human_review_reason`

Do not manufacture a top-pages table from ranked keywords alone. If a provider cannot supply page-level estimated traffic, include the ranking URL and label the traffic field `not_available`.

## Analysis rules

- Separate a direct business competitor from an organic search competitor. A publisher, marketplace, directory, or tool can compete for keywords without selling the same product.
- Treat abrupt changes as observations, not causal claims. Check the changed pages and keywords before suggesting a reason.
- Prefer page groups, intent, relevance, and trend direction over one headline traffic number.
- For every quantitative field, retain `provider`, `report_or_endpoint`, `retrieved_at_utc`, `market`, `language`, `scope`, and `metric_definition` in the relevant output or its data dictionary.
- Label data `not_available`, `not_comparable`, or `estimated` rather than guessing. Explain why it cannot be compared when possible.
- Do not claim that an SEO metric predicts ranking, traffic, revenue, conversion, market share, or AI citations.

## Required report

Write `competitor-traffic-report.md` with these sections:

1. **Executive answer:** target, scope, market, language, retrieval time, provider coverage, and the shortest useful conclusion.
2. **What the numbers mean:** which metrics are estimates, what they do and do not measure, and comparison limits.
3. **Traffic and visibility snapshot:** a provider-by-provider table. Do not merge values.
4. **Trend and change check:** observed movement, relevant changed pages/keywords, and confidence or gaps.
5. **Top pages:** a compact table with URL, apparent page role, leading keyword when available, estimated contribution, provider, and a human interpretation.
6. **Keyword and intent patterns:** group a limited set of relevant keywords by searcher job. Identify themes, not a dump of thousands of rows.
7. **Competitors and gaps:** distinguish direct competitors from organic competitors. If the user's domain is supplied, show only review-worthy content gaps and cannibalization risks.
8. **Prioritized actions:** no more than five actions. Each must cite the evidence, likely owner, expected output, and a human approval gate.
9. **Data availability and caveats:** unavailable providers, errors, scope mismatches, freshness limits, and exact recovery steps.
10. **Artifact index:** paths to generated CSVs and a field dictionary.

## Quality gates before finishing

- Verify that target scope is explicit: root domain, subdomain, path, or exact URL.
- Verify market and language are stated; highlight defaults.
- Verify provider connection details do not appear in research artifacts, report tables, CSV output, or the final summary.
- Verify every headline metric names its provider and says `estimated` when it is not first-party data.
- Verify provider values were not added, averaged, or compared across mismatched markets, languages, scopes, or dates.
- Verify all recommended actions are evidence-backed and require human approval before website changes or publishing.
- If data is unavailable, complete the report with the availability log and the exact next input needed; do not return a fictional analysis.

## Final response to the user

State the target, market, language, available providers, top three observations, most important caveat, generated artifact paths, and the one decision the user should make next.

$competitor-traffic-research
Проанализируй органический поисковый трафик competitor.com. Используй всех доступных поставщиков и рынок и язык по умолчанию. До начала перечисли типы отчётов, рынок, язык и возможное число тарифицируемых запросов.

$competitor-traffic-research
Мой сайт — mysite.com, конкурент — competitor.com. В одном рынке и на одном языке проверь пробелы органического контента. Оставь только возможности, связанные с B2B-системами управления проектами и заслуживающие ручной проверки. Пока не рекомендуй публикацию страниц; сначала подготовь исследовательский отчёт.