Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/jbms/finance-dl/llms.txt

Use this file to discover all available pages before exploring further.

finance-dl ships a command-line interface that lets you run any single scraper configuration directly. This page covers every argument accepted by finance_dl.cli, explains how the config module is resolved at runtime, and shows practical examples for both the --config and --spec invocation styles.

​
Basic Usage

You can invoke the CLI as a Python module or via the installed entry point: 
python -m finance_dl.cli --config-module <module> --config <name>
The <module> is the dotted Python import path to a file that contains one or more CONFIG_<name> functions, and <name> is the suffix that identifies which function to call.

​
How the Config Module Is Found

Before importing your config module, finance_dl.cli appends os.getcwd() to sys.path. This means you can place your config file in the current working directory and reference it by name without installing it as a package or manually adjusting PYTHONPATH. 
# Run from the directory that contains finance_dl_config.py
cd ~/finance
python -m finance_dl.cli --config-module finance_dl_config --config mybank

​
Flags Reference

​
--config-module
string
The importable Python module that defines your CONFIG_<name> functions. Must be reachable on sys.path — the current directory is added automatically, so you can run from the folder that contains the config file.
​
--config, -c
string
The configuration name to run. Resolves to the function CONFIG_<name> inside the config module. Mutually exclusive with --spec; exactly one of --config or --spec is required.
​
--spec, -s
string
A raw JSON string that fully specifies the scraper to run, without a config module. The JSON object must include a "module" key naming the scraper module to import. All other keys are forwarded as keyword arguments to that module’s run() function. Mutually exclusive with --config; exactly one of --spec or --config is required.Example:
--spec '{"module": "finance_dl.ofx", "ofx_params": {...}, "output_directory": "/data/vanguard"}'
​
--interactive, -i
boolean
default:"false"
Launch an IPython shell instead of running the scraper automatically. Implies --visible, so the browser window is always displayed in interactive mode. See Interactive Mode for details.
​
--visible
boolean
default:"false"
Run with a visible browser window. Useful for watching a scraper execute or for debugging login flows without dropping into a full interactive session. Implied automatically by --interactive.
​
--log
string
default:"INFO"
Python logging level name. Accepts DEBUG, INFO, WARNING, ERROR, or CRITICAL (case-insensitive). Controls verbosity of all log output to stdout.

​
How --spec Works

When you pass --spec, the CLI:
  1. Parses the value with json.loads.
  2. Pops the "module" key from the resulting dict to determine which scraper module to import.
  3. Sets headless to True by default (or False if --visible or --interactive is present), inserting it into the spec only if it is not already set.
  4. Calls module.run(**remaining_kwargs) with the rest of the dict.
This lets you run any scraper without maintaining a separate config file — handy for quick one-off runs or scripted pipelines.

​
Examples

# Runs CONFIG_mybank() from finance_dl_config.py, found in the current directory
python -m finance_dl.cli \
  --config-module finance_dl_config \
  --config mybank

Build docs developers (and LLMs) love

Get started for freeTalk to us