Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/SudharakaA/ai-job-search/llms.txt

Use this file to discover all available pages before exploring further.

The /scrape command turns your configured search queries into a curated list of new job postings. It loads your previous search state, runs parallel queries across job portals, fetches and parses each promising result, filters out anything already seen or already tracked, and presents only genuinely new matches ranked by fit. When you spot a role you want to pursue, you can transition directly into /apply from the results table without leaving the session.

​
How a scrape run works

1
Load state
2
Before making a single search request, the scraper loads two files:
3
  • job_scraper/seen_jobs.json — the full deduplication ledger for every URL or company-title key the scraper has encountered. If the file does not exist yet, it is initialized as {"seen": {}}.
  • job_search_tracker.csv — your application tracking spreadsheet. Any company-role combination already present here is skipped automatically, so jobs you have already applied to never resurface.
    • 4
    Reading both files first means deduplication is fast and requires no re-fetching.
    5
    Run WebSearch queries from search-queries.md
    6
    The scraper reads .claude/skills/job-scraper/search-queries.md for the full set of queries, then decides which subset to run:
    7
  • Default mode: runs the top 3 priority categories from the file.
  • Broad mode (/scrape broad): runs all query categories for the widest possible net.
  • Focus-area mode (/scrape data science, /scrape geophysics, etc.): prioritizes queries from the matching category.
    • 8
    Each priority category in search-queries.md maps to a specific career direction. Priority 1 covers your strongest and most desired role type, Priority 2 covers your domain expertise, Priority 3 covers adjacent pivot roles, and Priority 4 casts a broader technical net. /setup populates the file with your actual role titles, key skills, and location terms; you edit it directly as your priorities evolve, or re-run /setup --section search for a guided update.
    9
    Queries are structured as site-scoped searches — for example, site:jobindex.dk "Data Scientist" Copenhagen — so results stay anchored to real postings on specific portals. LinkedIn searches use the linkedin.com/jobs path, which is country-agnostic and accepts an explicit -l location flag.
    10
    The framework ships with CLI tools for Danish portals (Jobindex, Jobbank, Jobdanmark, Jobnet) and LinkedIn’s public job listings. If you are in a different market, run /add-portal to scaffold a skill for your local job boards using the same pattern.
    11
    Fetch and parse each promising result
    12
    For each search result that passes a title-and-snippet pre-filter, the scraper fetches the posting page and extracts:
    13
  • Job title and company name
  • Location and posting date (or "recent" when no date is available)
  • URL
  • Key requirements (brief list of skills and experience mentioned in the posting)
  • Application deadline if listed
    • 14
    The pre-filter step matters for efficiency: snippets and titles are evaluated before fetching, so low-relevance results never consume a full WebFetch call.
    15
    Quick fit assessment
    16
    For each new job, the scraper runs a fast three-tier assessment against your profile — not the full five-dimension scoring from 04-job-evaluation.md, but a quick signal:
    17
    RatingMeaningHighRole directly involves your core skills and target role typeMediumRole is adjacent to your experience with transferable skillsLowRole requires significant skills you lack
    18
    This is deliberately lightweight. The full fit evaluation (skills match, experience match, behavioral match, salary benchmark, and overall score) happens when you select a specific job and invoke /apply.
    19
    Deduplicate and store
    20
    All fetched jobs — including ones skipped because they were already seen — are written to seen_jobs.json with this structure:
    21
    {
  "seen": {
    "https://jobindex.dk/job/1234567": {
      "title": "Senior Data Scientist",
      "company": "Acme Analytics",
      "url": "https://jobindex.dk/job/1234567",
      "first_seen": "2025-01-15",
      "fit": "high",
      "status": "new"
    }
  }
}
    22
    Writing every result to the ledger (not just new ones) prevents them from being re-fetched and re-evaluated on every subsequent run. Jobs already present in seen_jobs.json or job_search_tracker.csv are skipped inline during the fetch-and-parse step, so this write pass finalises the record for all results encountered this run.
    23
    Present results
    24
    New jobs are presented in a table sorted by fit rating, high-match roles first:
    25
    ## New Job Matches - 2025-01-15

Found 8 new positions (3 high, 4 medium, 1 low match).

| # | Fit    | Title                     | Company          | Location   | Deadline   | URL          |
|---|--------|---------------------------|------------------|------------|------------|--------------|
| 1 | High   | Senior Data Scientist     | Acme Analytics   | Copenhagen | 2025-02-01 | [Link](...)  |
| 2 | High   | ML Engineer               | NordTech         | Copenhagen | 2025-01-28 | [Link](...)  |
| 3 | High   | Data Scientist – NLP      | Baltic AI        | Remote     | 2025-02-10 | [Link](...)  |
| 4 | Medium | Analytics Engineer        | Retail Group     | Aarhus     | 2025-01-31 | [Link](...)  |
    26
    For each high-match job, the scraper adds a short highlight block:
    27
  • Why the role matches your profile
  • Key requirements to check before applying
  • Any red flags worth noting (vague role description, unusual requirements, unclear seniority)
    • 28
    Transition to /apply
    29
    After presenting results, the scraper asks which jobs you want to evaluate in detail. Pick a number from the table and the full /apply workflow begins immediately: fit evaluation first, then CV and cover letter drafting if you confirm you want to proceed.
    30
    You can also supply a URL or pasted job description directly:
    31
    /apply https://jobindex.dk/job/1234567

    ​
    Configuring search-queries.md

    The file .claude/skills/job-scraper/search-queries.md controls everything the scraper searches for. It has four sections: Query categories (Priority 1-4): Each category is a set of site-scoped WebSearch queries. Queries combine a job title or skill keyword with a location term. Priority 1 should contain your most desired role type; Priority 4 is the broad fallback. Location filter: Defines acceptable commute areas. The scraper skips results that fall outside the ranges you specify here, so you never see jobs that require relocation or an impossible commute. Date filter: Only postings from the last 14 days are included. Postings with an expired deadline are also skipped. If a date cannot be determined, the result is included but flagged as "date unknown". Adapting for focus areas: When you pass a focus-area argument, the scraper prioritizes queries from the matching category.
    Run /setup --section search to reconfigure your search queries through a guided interview. This is useful when your priorities shift or you want to add new role types without re-doing your full profile.

    ​
    Scraping rules

    The scraper only presents jobs found through actual WebSearch and WebFetch results. It does not synthesize, guess, or hallucinate job listings.
    Where possible, the scraper runs multiple WebSearch queries in parallel using the Agent tool. This keeps a typical run fast even when covering all priority categories.
    Results outside the commute range defined in your location filter are skipped automatically. You do not need to manually filter for location.
    Postings with expired deadlines or those explicitly marked as closed are skipped. The scraper does not re-surface closed roles on subsequent runs.
    Titles and snippets from search results are evaluated before a full WebFetch is made. Low-relevance results are pre-filtered to keep the scrape fast and avoid unnecessary fetches.

    Build docs developers (and LLMs) love

    Get started for freeTalk to us