diff --git a/.ai/ARCHITECTURE.md b/.ai/ARCHITECTURE.md new file mode 100644 index 0000000000..ca6816ccf9 --- /dev/null +++ b/.ai/ARCHITECTURE.md @@ -0,0 +1,775 @@ +# Architecture + +## Python Backend Framework + +- **`dash/dash.py`** - Main `Dash` application class (~2000 lines). Orchestrates Flask server, layout management, callback registration, routing, and asset serving. Key methods: `layout` property, `callback()`, `clientside_callback()`, `run()`. + +- **`dash/_callback.py`** - Callback registration and execution. Contains `callback()` decorator (usable as `@dash.callback` without app instance), `clientside_callback()`, and `register_callback()` which inserts callbacks into the callback map. + +- **`dash/dependencies.py`** - Dependency classes for callbacks: + - `Input` - Triggers callback when value changes + - `Output` - Component property to update (supports `allow_duplicate=True`) + - `State` - Read value without triggering callback + - `ClientsideFunction` - Reference to JS function for clientside callbacks + - Wildcards: `MATCH`, `ALL`, `ALLSMALLER` for pattern-matching IDs + +- **`dash/development/base_component.py`** - `Component` base class with `ComponentMeta` metaclass. All Dash components inherit from this. Components auto-register in `ComponentRegistry` and serialize to JSON via `to_plotly_json()`. + +- **`dash/_pages.py`** - Multi-page app support. `PAGE_REGISTRY` holds registered pages, `register_page()` decorator registers page modules with routes. + +## Layout System + +The layout defines the UI as a tree of components: + +```python +app.layout = html.Div([ + dcc.Input(id='input', value='initial'), + html.Div(id='output') +]) +``` + +- **Static layout**: Assigned directly as a component tree +- **Dynamic layout**: Assigned as a function that returns components (called on each page load, useful for per-session state) +- Layout is serialized to JSON and sent to the React frontend via `/_dash-layout` +- Components can contain other components via `children` prop +- Component IDs can be strings or dicts (for pattern-matching callbacks) + +## Callback Types + +### 1. Regular Callbacks + +`@app.callback` or `@dash.callback`: + +```python +@app.callback(Output('output', 'children'), Input('input', 'value')) +def update(value): + return f'You entered: {value}' +``` + +Server-side Python function called when inputs change. Outputs update component properties. + +### 2. Clientside Callbacks + +`app.clientside_callback`: + +```python +app.clientside_callback( + """function(value) { return 'You entered: ' + value; }""", + Output('output', 'children'), + Input('input', 'value') +) +``` + +JavaScript function runs in browser. Faster for simple transformations, no server round-trip. Can reference `window.dash_clientside.namespace.function_name` or inline JS string. + +### 3. Background Callbacks + +`background=True`: + +```python +@app.callback(Output('output', 'children'), Input('btn', 'n_clicks'), + background=True, manager=diskcache_manager, + running=[(Output('btn', 'disabled'), True, False)], + progress=[Output('progress', 'value')]) +def compute(set_progress, n_clicks): + for i in range(10): + set_progress(i * 10) + time.sleep(1) + return 'Done' +``` + +Callbacks executed in separate process via Celery or Diskcache manager. Supports `progress` updates, `running` state changes, and `cancel` inputs. See [Background Callbacks](#background-callbacks) section for details. + +### 4. Pattern-Matching Callbacks + +```python +@app.callback( + Output({'type': 'output', 'index': MATCH}, 'children'), + Input({'type': 'input', 'index': MATCH}, 'value') +) +def update(value): + return value +``` + +Use dict IDs with wildcards (`MATCH`, `ALL`, `ALLSMALLER`) to target dynamically-generated components. + +## Server Routes + +- `/_dash-layout` - Returns initial component tree as JSON +- `/_dash-dependencies` - Returns callback definitions +- `/_dash-update-component` - Executes callbacks, returns updated props +- `/_dash-component-suites//` - Serves component JS/CSS assets +- `/assets/` - Serves static assets from app's assets folder + +## Frontend (dash-renderer) + +**`dash/dash-renderer/src/`** contains the TypeScript/React frontend. See [RENDERER.md](RENDERER.md) for detailed documentation on: + +- Layout traversal (`crawlLayout`) and `children_props` +- Component resolution from `window[namespace][type]` +- Callback triggering via `setProps` and `notifyObservers` +- Redux store structure (layout, paths, callbacks, graphs) +- Observer system for callback processing +- `window.dash_clientside` API +- `window.dash_component_api` API + +### React Version + +Dash supports multiple React versions. Configured in `dash/_dash_renderer.py`. + +**Available versions:** 18.3.1 (default), 18.2.0, 16.14.0 + +Set via environment variable (experimental): + +```bash +REACT_VERSION=16.14.0 python app.py +``` + +Or programmatically before creating the app: + +```python +from dash._dash_renderer import _set_react_version +_set_react_version("16.14.0") + +from dash import Dash +app = Dash(__name__) +``` + +This is useful for compatibility with older component libraries that require React 16. + +## Pages System + +Multi-page apps use `dash/_pages.py` with automatic routing via `dcc.Location`. + +### Page Registration + +Each page module calls `register_page()`: + +```python +# pages/analytics.py +from dash import register_page, html + +register_page(__name__) # infers path /analytics from module name + +layout = html.Div("Analytics page") +``` + +- **`PAGE_REGISTRY`** - `OrderedDict` storing all registered pages with metadata +- **`register_page(module, path=None, ...)`** - Registers page with inferred or explicit path, title, description, image + +### Page Container + +When `use_pages=True`, Dash injects `page_container` as the layout (`dash/dash.py:148-158`): + +```python +page_container = html.Div([ + dcc.Location(id="_pages_location", refresh="callback-nav"), + html.Div(id="_pages_content"), # current page layout injected here + dcc.Store(id="_pages_store"), # stores page title/metadata +]) +``` + +### Routing Mechanism + +1. `dcc.Location` tracks browser URL changes +2. Internal callback listens to `pathname` and `search` inputs +3. `_path_to_page()` matches URL to registered page in `PAGE_REGISTRY` +4. Page layout injected into `_pages_content` div + +### Path Templates (Dynamic Routes) + +Pages can capture URL variables: + +```python +register_page(__name__, path_template="/asset/") + +def layout(asset_id=None): + return html.Div(f"Asset: {asset_id}") +``` + +`_parse_path_variables()` extracts variables via regex and passes them as kwargs to the layout function. + +### Auto-Discovery + +`_import_layouts_from_pages()` walks the `pages/` folder: +- Skips files starting with `_` or `.` +- Only imports `.py` files containing `register_page` +- Auto-assigns `layout` attribute from each module to the registry + +### Page Ordering + +Pages sorted by: numeric `order` → string `order` → no order → module name. Home page (`/`) defaults to order `0`. + +## Assets and Static Files + +### Asset Directory + +The `assets/` folder is automatically scanned at startup (`dash/dash.py:_walk_assets_directory`): + +- `.css` files → appended to stylesheets +- `.js` files → appended to scripts +- `favicon.ico` → used as app favicon +- Files matching `assets_ignore` regex are skipped + +### Loading Order + +Resources load in this order (`dash/dash.py:1127-1165`): + +1. React dependencies (from dash-renderer) +2. Component library scripts (dash-html-components, dash-core-components, etc.) +3. External scripts (`external_scripts` parameter) +4. Dash renderer bundle +5. Clientside callback scripts (inline) + +CSS follows similar ordering with external stylesheets first. + +### Fingerprinting and Caching + +Component assets use fingerprinted URLs for cache busting (`dash/fingerprint.py`): + +``` +/_dash-component-suites/dash_core_components/dash_core_components.v2_14_0m1699900000.min.js +``` + +- Fingerprinted resources: 1-year cache header +- Non-fingerprinted: ETag validation +- Asset files: query string `?m={modification_time}` + +### Configuration Options + +```python +Dash( + assets_folder='assets', # path to assets directory + assets_url_path='assets', # URL path segment + assets_ignore='.*ignored.*', # regex to skip files + assets_external_path=None, # CDN base URL for assets + serve_locally=True, # True=local files, False=CDN + external_scripts=[], # additional JS URLs + external_stylesheets=[], # additional CSS URLs +) +``` + +### Asset URL Generation + +`app.get_asset_url(path)` returns the correct URL accounting for `requests_pathname_prefix` (important for Dash Enterprise deployments where apps have URL prefixes). + +## Error Handling + +### Debug Mode + +Debug mode enables developer tools (`dash/dash.py:_setup_dev_tools`): + +```python +app.run(debug=True) +# Or via environment: DASH_DEBUG=true +``` + +### Dev Tools Options + +```python +app.enable_dev_tools( + dev_tools_ui=True, # show error UI overlay + dev_tools_props_check=True, # validate component prop types + dev_tools_serve_dev_bundles=True, # use development JS (better errors) + dev_tools_hot_reload=True, # auto-reload on file changes + dev_tools_prune_errors=True, # strip internal frames from tracebacks +) +``` + +Environment variables: `DASH_DEBUG`, `DASH_UI`, `DASH_PROPS_CHECK`, `DASH_HOT_RELOAD`, etc. + +### Callback Exceptions + +**`PreventUpdate`** - Skip updating outputs without error: + +```python +from dash.exceptions import PreventUpdate + +@app.callback(Output('out', 'children'), Input('in', 'value')) +def update(value): + if not value: + raise PreventUpdate + return value +``` + +**`no_update`** - Skip specific outputs in multi-output callbacks: + +```python +from dash import no_update + +@app.callback(Output('a', 'children'), Output('b', 'children'), Input('in', 'value')) +def update(value): + return value, no_update # only updates 'a' +``` + +### Error Handlers + +Callbacks support `on_error` for custom error handling: + +```python +def handle_error(err): + logging.error(f"Callback failed: {err}") + return "Error occurred" # returned to output + +@app.callback(Output('out', 'children'), Input('in', 'value'), on_error=handle_error) +def update(value): + return 1 / 0 # triggers error handler +``` + +App-level error handler set via constructor. + +### Validation + +- **Layout validation**: When `suppress_callback_exceptions=False` (default), checks that callback IDs exist in layout +- **Callback validation**: `dev_tools_validate_callbacks=True` checks for circular dependencies +- **Props checking**: Validates component prop types against schema in dev mode + +### Hot Reload + +When enabled, a watch thread monitors: +- `assets/` folder for CSS/JS changes +- Component package directories + +Frontend polls `/_reload-hash` and triggers reload when hash changes. Configurable via `hot_reload_interval` (default 3s) and `hot_reload_watch_interval` (default 0.5s). + +## Background Callbacks + +Background callbacks execute in separate processes, allowing the main server to remain responsive. Managed by `dash/background_callback/managers/`. + +### Definition + +```python +from dash import callback, Input, Output +from dash.background_callback import DiskcacheManager + +cache_manager = DiskcacheManager() + +@callback( + Output("result", "children"), + Input("button", "n_clicks"), + background=True, + manager=cache_manager, + interval=500, # polling interval in ms +) +def compute(n_clicks): + # Expensive computation + return result +``` + +### Callback Managers + +**`DiskcacheManager`** (`dash/background_callback/managers/diskcache_manager.py`): +- Uses `diskcache.Cache` for persistent storage +- Spawns `multiprocess.Process` for each job +- Results stored on disk, survives server restarts +- Good for single-server deployments + +**`CeleryManager`** (`dash/background_callback/managers/celery_manager.py`): +- Requires Celery app with result backend (Redis/RabbitMQ) +- Jobs distributed across Celery workers +- Supports horizontal scaling +- Good for production multi-worker deployments + +```python +from celery import Celery +from dash.background_callback import CeleryManager + +celery_app = Celery(__name__, broker="redis://localhost:6379/0") +cache_manager = CeleryManager(celery_app) +``` + +### Progress Updates + +The `progress` parameter defines outputs updated during execution: + +```python +@callback( + Output("result", "children"), + Input("button", "n_clicks"), + progress=Output("progress-bar", "value"), + progress_default=0, + background=True, + manager=cache_manager, +) +def compute(set_progress, n_clicks): + for i in range(100): + set_progress(i) + time.sleep(0.1) + return "Complete" +``` + +- `set_progress` is injected as first argument when `progress` is specified +- Can be single Output or list of Outputs +- `progress_default` sets value when callback not running + +### Running State + +The `running` parameter updates outputs while the job executes: + +```python +@callback( + Output("result", "children"), + Input("button", "n_clicks"), + running=[ + (Output("button", "disabled"), True, False), + (Output("status", "children"), "Computing...", "Ready"), + ], + background=True, + manager=cache_manager, +) +def compute(n_clicks): + time.sleep(5) + return "Done" +``` + +Each tuple: `(Output, value_while_running, value_when_complete)` + +### Cancellation + +The `cancel` parameter specifies inputs that abort the job: + +```python +@callback( + Output("result", "children"), + Input("start-btn", "n_clicks"), + cancel=[Input("cancel-btn", "n_clicks")], + background=True, + manager=cache_manager, +) +def compute(n_clicks): + # Job terminates if cancel-btn clicked + return result +``` + +Managers call `terminate_job()` which kills the process (Diskcache) or revokes the task (Celery). + +### Result Caching + +Results can be cached to avoid recomputation: + +```python +def get_user_id(): + return flask.session.get("user_id") + +cache_manager = DiskcacheManager( + cache_by=[get_user_id], # cache key includes user ID + expire=3600, # TTL in seconds +) +``` + +- `cache_by` - List of functions whose return values are included in cache key +- `expire` - Time-to-live for cached results +- `cache_args_to_ignore` - Argument indices to exclude from cache key + +### How It Works + +1. **Initial request**: Frontend triggers callback, backend returns `cacheKey` and `job` ID +2. **Polling**: Frontend polls `/_dash-update-component?cacheKey=...&job=...` at configured interval +3. **Progress**: Each poll returns current progress value if set +4. **Completion**: When job finishes, poll returns final result +5. **Cleanup**: Results cleared from cache (unless `cache_by` specified) + +Cache key is SHA256 hash of: function source + arguments + triggered inputs + cache_by values. + +### Key Files + +- `dash/_callback.py:188-219` - Background spec construction +- `dash/background_callback/managers/__init__.py` - `BaseBackgroundCallbackManager` abstract class +- `dash/background_callback/managers/diskcache_manager.py` - Diskcache implementation +- `dash/background_callback/managers/celery_manager.py` - Celery implementation +- `dash/dash-renderer/src/actions/callbacks.ts:458-685` - Frontend polling logic + +## Jupyter Integration + +Dash apps can run directly in Jupyter notebooks and JupyterLab. The integration is handled by `dash/_jupyter.py`. + +### Display Modes + +```python +app.run( + jupyter_mode="inline", # Display in notebook cell (default) + jupyter_width="100%", # IFrame width + jupyter_height=650, # IFrame height in pixels +) +``` + +| Mode | Behavior | +|------|----------| +| `"inline"` | App displays in notebook cell via IFrame | +| `"external"` | Prints URL, user opens in browser tab | +| `"jupyterlab"` | Opens in dedicated JupyterLab tab | +| `"tab"` | Auto-opens URL in new browser tab | + +### How It Works + +1. `app.run()` detects Jupyter environment via `get_ipython()` +2. Flask server starts in background daemon thread +3. Jupyter comm protocol negotiates proxy configuration +4. App displays according to selected mode + +``` +app.run() in notebook + ↓ +Detect Jupyter → Start Flask in background thread + ↓ +Comm request → Extension responds with base_url + ↓ +Compute dashboard URL with proxy path + ↓ +Display: IFrame (inline) / URL (external) / Tab (jupyterlab) +``` + +### Notebook Extension + +Classic Jupyter notebooks use `dash/nbextension/`: + +- `main.js` - Registers "dash" comm target +- `dash.json` - Extension loader configuration + +The extension handles comm messages: +- `base_url_request` → responds with server URL and base path +- Enables proper proxy routing in JupyterHub environments + +### JupyterLab Extension + +JupyterLab uses `@plotly/dash-jupyterlab/`: + +- `src/index.ts` - TypeScript plugin implementing `JupyterFrontEndPlugin` +- `DashIFrameWidget` - Lumino widget for rendering apps in tabs + +Handles messages: +- `base_url_request` → responds with JupyterLab server config +- `show` → creates dedicated tab with IFrame widget + +Compatible with JupyterLab 2.x, 3.x, and 4.x. + +### Proxy Configuration + +In JupyterHub/proxy environments, the extension negotiates `requests_pathname_prefix`: + +```python +# Computed from Jupyter base path +requests_pathname_prefix = "/user/username/proxy/8050/" +``` + +This ensures callbacks route correctly through the Jupyter proxy. + +### Google Colab + +Special handling for Colab: +- Uses `google.colab.output.serve_kernel_port_as_iframe()` for inline +- Uses `google.colab.output.serve_kernel_port_as_window()` for external +- Only supports "inline" and "external" modes + +### Key Files + +- `dash/_jupyter.py` - `JupyterDash` class, comm handling, server thread +- `dash/nbextension/main.js` - Classic notebook extension +- `@plotly/dash-jupyterlab/src/index.ts` - JupyterLab extension + +## Configuration Reference + +### Dash() Constructor Parameters + +**Basic Setup:** +- `name` - Flask app name (default: infers from `__name__`) +- `server` - Flask instance or `True` to create new (default: `True`) +- `title` - Browser tab title (default: `"Dash"`) +- `update_title` - Title during callbacks (default: `"Updating..."`) + +**Assets & Resources:** +- `assets_folder` - Path to assets directory (default: `"assets"`) +- `assets_url_path` - URL path for assets (default: `"assets"`) +- `assets_ignore` - Regex to exclude assets (default: `""`) +- `serve_locally` - Serve from local vs CDN (default: `True`) +- `external_scripts` - Additional JS URLs +- `external_stylesheets` - Additional CSS URLs + +**Routing:** +- `url_base_pathname` - Base URL prefix for entire app +- `requests_pathname_prefix` - Prefix for AJAX requests +- `routes_pathname_prefix` - Prefix for API routes + +**Multi-Page:** +- `use_pages` - Enable pages system (default: auto-detect) +- `pages_folder` - Path to pages directory (default: `"pages"`) + +**Behavior:** +- `suppress_callback_exceptions` - Skip callback validation (default: `False`) +- `prevent_initial_callbacks` - Skip callbacks on load (default: `False`) +- `background_callback_manager` - DiskcacheManager or CeleryManager +- `on_error` - Global callback error handler + +### app.run() Parameters + +- `host` - Server IP (default: `"127.0.0.1"`, env: `HOST`) +- `port` - Server port (default: `8050`, env: `PORT`) +- `debug` - Enable dev tools (default: `False`, env: `DASH_DEBUG`) +- `jupyter_mode` - Display mode: `"inline"`, `"external"`, `"tab"` + +### Environment Variables + +| Variable | Purpose | +|----------|---------| +| `DASH_DEBUG` | Enable debug mode | +| `DASH_URL_BASE_PATHNAME` | Base URL prefix | +| `DASH_SUPPRESS_CALLBACK_EXCEPTIONS` | Skip validation | +| `DASH_HOT_RELOAD` | Enable hot reload | +| `DASH_PROPS_CHECK` | Validate prop types | +| `DASH_PRUNE_ERRORS` | Simplify tracebacks | +| `HOST` | Server host | +| `PORT` | Server port | + +## Stores and Client-Side State + +### dcc.Store + +Store data client-side with configurable persistence: + +```python +dcc.Store(id='my-store', storage_type='local', data={'key': 'value'}) +``` + +| Storage Type | Persists | Scope | Use Case | +|--------------|----------|-------|----------| +| `'memory'` | Page view only | Tab | Temporary state, debugging | +| `'session'` | Browser session | Tab | Form state, filters | +| `'local'` | Forever | All tabs | User preferences, settings | + +**Usage pattern:** +```python +@app.callback(Output('output', 'children'), Input('store', 'data')) +def use_store(data): + return data['key'] + +@app.callback(Output('store', 'data'), Input('input', 'value')) +def update_store(value): + return {'key': value} +``` + +### Component Persistence + +Automatically persist user edits to component props: + +```python +dcc.Dropdown( + id='dropdown', + options=[...], + persistence=True, # Enable persistence + persistence_type='local', # local, session, or memory + persisted_props=['value'], # Props to persist (default varies by component) +) +``` + +- **`persistence`** - `True` or unique key to enable +- **`persistence_type`** - Storage backend (default: `'local'`) +- **`persisted_props`** - List of prop names to persist + +Supported components: Input, Dropdown, Checklist, RadioItems, Slider, RangeSlider, DatePickerSingle, DatePickerRange, Textarea, Tabs, DataTable. + +### When to Use Each + +| Need | Solution | +|------|----------| +| Server-controlled state | `dcc.Store` with callbacks | +| Remember user selections | Component `persistence=True` | +| Share state across tabs | `dcc.Store` with `storage_type='local'` | +| Session-only state | `persistence_type='session'` | + +## Async Callbacks + +Dash supports `async def` callbacks for non-blocking execution. + +### Setup + +```bash +pip install dash[async] +``` + +Async is auto-enabled when `asgiref` is detected. Or explicitly: + +```python +app = Dash(__name__, use_async=True) +``` + +### Usage + +```python +import asyncio + +@app.callback(Output('output', 'children'), Input('input', 'value')) +async def async_update(value): + await asyncio.sleep(1) # Non-blocking + return f"Processed: {value}" +``` + +### Key Points + +- Regular async callbacks are **non-blocking** - multiple can run concurrently +- Background callbacks also support `async def` +- Jupyter uses `nest_asyncio` for event loop compatibility +- Without `dash[async]`, coroutines raise an error + +### Async with Background Callbacks + +```python +@app.callback( + Output('result', 'children'), + Input('btn', 'n_clicks'), + background=True, + manager=diskcache_manager, +) +async def async_background(n_clicks): + await asyncio.sleep(5) + return "Done" +``` + +Both DiskcacheManager and CeleryManager support async functions via `asyncio.run()`. + +## Security + +### XSS Protection + +Dash automatically sanitizes dangerous URLs in components: + +- Blocked protocols: `javascript:`, `vbscript:` +- Protected attributes: `href`, `src`, `action`, `formAction` +- Dangerous URLs replaced with `about:blank` + +Components with URL sanitization: `html.A`, `html.Form`, `html.Iframe`, `html.Embed`, `html.Object`, `html.Button`. + +### Content Security Policy (CSP) + +Generate hashes for inline scripts to use with CSP middleware: + +```python +from flask_talisman import Talisman + +Talisman(app.server, content_security_policy={ + "default-src": "'self'", + "script-src": ["'self'"] + app.csp_hashes() +}) +``` + +`app.csp_hashes(hash_algorithm='sha256')` returns base64-encoded hashes. + +### Callback Security + +- **`suppress_callback_exceptions=False`** (default) - Validates all callback IDs exist in layout +- **`prevent_initial_callbacks=True`** - Prevents callbacks firing on page load (can also set per-callback with `prevent_initial_call`) + +### Meta Tag Sanitization + +Meta tag values are HTML-escaped to prevent injection: + +```python +app = Dash(__name__, meta_tags=[ + {"name": "description", "content": "Safe "} +]) +``` + +### Key Files + +- `dash/dash-renderer/src/utils/clientsideFunctions.ts` - URL sanitization (`clean_url`) +- `dash/dash.py:csp_hashes()` - CSP hash generation +- `tests/integration/security/` - Security test coverage diff --git a/.ai/COMMANDS.md b/.ai/COMMANDS.md new file mode 100644 index 0000000000..4c155d87b7 --- /dev/null +++ b/.ai/COMMANDS.md @@ -0,0 +1,81 @@ +# Commands + +## Initial Setup + +```bash +# Create and activate virtual environment +python3 -m venv venv +source venv/bin/activate # Windows: source venv/scripts/activate + +# Install Python dependencies +pip install -e .[ci,dev,testing,celery,diskcache] + +# Install Node dependencies +npm ci +``` + +## Building + +```bash +# Full build (Linux/Mac) +npm run build + +# Full build (Windows - use Bash terminal, not PowerShell/CMD) +npm run first-build + +# Build single component after changes +dash-update-components "dash-core-components" # or dash-html-components, dash-table + +# Build renderer only +cd dash/dash-renderer && renderer build +``` + +## Testing + +Tests use pytest with Selenium/ChromeDriver. ChromeDriver must match your Chrome version. See [TESTING.md](TESTING.md) for fixtures, patterns, and detailed documentation. + +```bash +# Run all tests +npm run test + +# Unit tests only +pytest tests/unit + +# Integration tests (requires ChromeDriver) +pytest tests/integration + +# Run specific test by name +pytest -k test_name + +# Run tests matching pattern +pytest -k cbcx # runs all tests with "cbcx" in name + +# Renderer unit tests (Jest) +cd dash/dash-renderer && npm run test + +# Setup test components before running integration tests +npm run setup-tests.py +``` + +## Linting + +Linting runs automatically on commit via husky pre-commit hook and lint-staged (`.lintstagedrc.js`). You typically don't need to run these manually. + +**Pre-commit runs on staged files:** +- Python (`dash/`, `tests/`): pylint, flake8, black --check +- JavaScript/TypeScript: eslint, prettier --check (per component package) + +**Manual commands** (if needed): + +```bash +# Run all linters +npm run lint + +# Individual linters +npm run private::lint.black # Check Black formatting +npm run private::lint.flake8 # Flake8 +npm run private::lint.pylint-dash # Pylint on dash/ + +# Auto-format Python with Black +npm run private::format.black +``` diff --git a/.ai/COMPONENTS.md b/.ai/COMPONENTS.md new file mode 100644 index 0000000000..6d76b6a202 --- /dev/null +++ b/.ai/COMPONENTS.md @@ -0,0 +1,182 @@ +# Component System + +The component system bridges React components to Python with auto-generated wrappers. + +## Generation Pipeline + +``` +React/TypeScript Source + ↓ + extract-meta.js (Node.js) + ├── react-docgen (for .js/.jsx - parses PropTypes) + └── TypeScript Compiler API (for .tsx - parses type definitions) + ↓ + metadata.json + ↓ + dash-generate-components (Python CLI) + ↓ + Python component classes (+ R/Julia if requested) +``` + +### Key Files + +- **`dash/extract-meta.js`** - Node.js script that extracts component metadata. For JavaScript components, uses `react-docgen` to parse PropTypes. For TypeScript components, uses the TypeScript Compiler API to parse type definitions and convert them to metadata format. + +- **`dash/development/component_generator.py`** - CLI entry point (`dash-generate-components`). Orchestrates metadata extraction and code generation. + +- **`dash/development/_py_components_generation.py`** - Generates Python class files from metadata. Creates typed `__init__` methods, docstrings, and prop validation. + +- **`dash/development/_py_prop_typing.py`** - Maps JavaScript/TypeScript types to Python types (e.g., `arrayOf` → `typing.Sequence`, `shape` → `TypedDict`). + +- **`dash/development/_generate_prop_types.py`** - For TypeScript components, generates a `proptypes.js` file since TSX doesn't have runtime PropTypes. + +## Component JSON Structure + +Components serialize to `{type, namespace, props}` via `to_plotly_json()` in `base_component.py`: + +```python +# Python component +html.Div(id='my-div', children='Hello') + +# Serializes to JSON +{ + "type": "Div", + "namespace": "dash_html_components", + "props": { + "id": "my-div", + "children": "Hello" + } +} +``` + +This JSON is sent to the frontend via `/_dash-layout` (initial load) and `/_dash-update-component` (callback responses). + +## Frontend Component Resolution + +Components must be available on the frontend at `window[namespace][type]`: + +```javascript +// Component packages register themselves +window.dash_html_components = { + Div: DivComponent, + Span: SpanComponent, + // ... +}; + +window.dash_core_components = { + Dropdown: DropdownComponent, + Graph: GraphComponent, + // ... +}; +``` + +The renderer resolves components via `registry.js`: + +```javascript +resolve: (component) => { + const {type, namespace} = component; + return window[namespace][type]; // Returns React component class +} +``` + +## Package Structure + +### `_imports_.py` + +Auto-generated by `generate_imports()` in `_py_components_generation.py`: + +```python +from .Dropdown import Dropdown +from .Graph import Graph +from .Input import Input +# ... one import per component + +__all__ = [ + "Dropdown", + "Graph", + "Input", + # ... +] +``` + +### `__init__.py` + +Manually maintained, imports from `_imports_.py`: + +```python +from ._imports_ import * # Re-exports all components +from ._imports_ import __all__ as _components + +# Read version from package-info.json +with open(os.path.join(_basepath, "package-info.json")) as f: + package = json.load(f) +__version__ = package["version"] + +# Define JavaScript assets to serve +_js_dist = [ + { + "relative_package_path": "dash_core_components.js", + "namespace": "dash", + }, + # async chunks, source maps, proptypes.js for dev, etc. +] + +# Attach _js_dist to each component class +for _component in _components: + setattr(locals()[_component], "_js_dist", _js_dist) +``` + +## Resource System (`_js_dist` / `_css_dist`) + +**`dash/resources.py`** manages JavaScript and CSS asset loading for components. + +### Resource Entry Structure + +```python +{ + "relative_package_path": "dcc/dash_core_components.js", # Path within package + "external_url": "https://unpkg.com/...", # CDN fallback + "namespace": "dash", # JS namespace + "async": True | "eager" | "lazy", # Async loading mode + "dynamic": True, # Loaded on demand (source maps) + "dev_package_path": "dcc/proptypes.js", # Dev-only path + "dev_only": True, # Only in dev mode +} +``` + +### Resource Loading Flow + +1. Each component class has `_js_dist` (and optionally `_css_dist`) attribute set in `__init__.py` +2. When component is imported, `ComponentMeta` registers module in `ComponentRegistry.registry` +3. `ComponentRegistry.get_resources("_js_dist")` iterates registered modules, collects all `_js_dist` lists +4. `Scripts` / `Css` classes in `resources.py` filter resources based on config: + - `serve_locally=True`: Use `relative_package_path`, serve via `/_dash-component-suites/` + - `serve_locally=False`: Use `external_url` (CDN) + - `eager_loading=True`: Load async resources immediately + - `dev_bundles=True`: Include `dev_package_path` resources + +### Async Loading Modes + +- `async: True` - Dynamic unless `eager_loading` is enabled +- `async: "lazy"` - Always loaded dynamically (on-demand) +- `async: "eager"` - Loaded dynamically only if server isn't in eager mode + +## Creating New Components + +1. Write React component with PropTypes (JS) or TypeScript props interface (TSX) +2. Run `dash-generate-components src/lib/components -p package_name` +3. Generated Python wrapper goes to `package_name/ComponentName.py` +4. `_imports_.py` is auto-generated with imports for all components +5. For TSX, `proptypes.js` is also generated for runtime prop validation +6. Bundle with webpack, register on `window[namespace]` +7. Update `__init__.py` to set `_js_dist` on components + +## Built-in Component Packages + +Managed as a Lerna monorepo in `components/`: + +- **`components/dash-core-components/`** - Interactive components (Dropdown, Slider, Graph, Input, etc.) +- **`components/dash-html-components/`** - HTML element wrappers (Div, Span, H1, etc.) +- **`components/dash-table/`** - DataTable component (deprecated in favor of dash-ag-grid) + +Use `dash-update-components "component-name"` to rebuild after changes. diff --git a/.ai/README.md b/.ai/README.md new file mode 100644 index 0000000000..7d94909213 --- /dev/null +++ b/.ai/README.md @@ -0,0 +1,48 @@ +# Dash AI Agent Guide + +This directory contains documentation for AI coding assistants working with the Dash codebase. + +## Quick Links + +- [Commands](./COMMANDS.md) - Build, test, and lint commands +- [Architecture](./ARCHITECTURE.md) - Backend, callbacks, pages, assets, errors, background callbacks, Jupyter, config, stores, async, security +- [Renderer](./RENDERER.md) - Frontend, crawlLayout, Redux store, clientside API, component API +- [Components](./COMPONENTS.md) - Component generation, package structure, resource system +- [Testing](./TESTING.md) - Testing framework, fixtures, patterns, type compliance +- [Troubleshooting](./TROUBLESHOOTING.md) - Common errors and solutions + +## Project Overview + +Dash is a Python framework for building reactive web-based data visualization applications. Built on Plotly.js, React, and Flask, it ties UI elements (dropdowns, sliders, graphs) directly to analytical Python code. + +## Key Directories + +``` +dash/ +├── dash/ # Main Python package +│ ├── dash.py # Core Dash app class +│ ├── _callback.py # Callback registration/execution +│ ├── dependencies.py # Input/Output/State classes +│ ├── _pages.py # Multi-page app support +│ ├── development/ # Component generation tools +│ ├── dash-renderer/ # TypeScript/React frontend +│ └── dcc/, html/, dash_table/ # Built-in components +├── components/ # Component source packages (Lerna monorepo) +│ ├── dash-core-components/ +│ ├── dash-html-components/ +│ └── dash-table/ +├── tests/ +│ ├── unit/ # pytest unit tests +│ ├── integration/ # Selenium browser tests +│ ├── compliance/ # Type checking (pyright/mypy) +│ └── background_callback/ # Background callback tests +└── requirements/ # Modular dependency files +``` + +## Code Review Conventions + +Emoji used in reviews: +- `:dancer:` - Can merge +- `:tiger2:` - Needs more tests +- `:snake:` - Security concern +- `:pill:` - Performance issue diff --git a/.ai/RENDERER.md b/.ai/RENDERER.md new file mode 100644 index 0000000000..d258f49899 --- /dev/null +++ b/.ai/RENDERER.md @@ -0,0 +1,317 @@ +# Dash Renderer + +The dash-renderer is the TypeScript/React frontend that powers Dash applications. Located in `dash/dash-renderer/src/`. + +## Initialization Flow + +``` +1. DashRenderer constructor + └─ ReactDOM.createRoot('#react-entry-point') + └─ + +2. AppProvider creates Redux store + └─ Registers observers for callback processing + +3. APIController fetches from server + ├─ GET /_dash-layout → component tree + ├─ GET /_dash-dependencies → callback definitions + └─ Dispatches setLayout, setGraphs, setPaths + +4. Config includes children_props from ComponentRegistry + └─ Stored in window.__dashprivate_childrenProps + +5. hydrateInitialOutputs() + ├─ Validates callbacks against layout + ├─ Triggers initial callbacks + └─ Sets appLifecycle: 'HYDRATED' + +6. DashWrapper renders component tree +``` + +## Layout Traversal (crawlLayout) + +The layout is traversed using `crawlLayout` (`actions/utils.js`). This algorithm: + +1. **For arrays**: Iterates each child, following extra paths for nested components +2. **For objects (components)**: + - Applies the visitor function to the component + - Follows `props.children` if present + - Follows additional paths from `children_props` config + +### children_props + +Each component class defines `_children_props` listing props that contain nested components. This is: +1. Generated from React component PropTypes/TypeScript during component generation +2. Stored on the Python component class as `_children_props` +3. Collected into `ComponentRegistry.children_props` when components are imported +4. Sent to frontend via config (`dash.py:933`) +5. Stored in `window.__dashprivate_childrenProps` on the frontend + +```python +# Example: Python component class +class Dropdown(Component): + _children_props = ['options.[].label', 'options.[].title'] + # ... +``` + +### Pattern Syntax + +| Pattern | Meaning | Example | +|---------|---------|---------| +| `children` | Direct children prop | Standard | +| `prop.[]` | Array items are components | `options.[]` | +| `prop.[].sub` | `sub` prop of array items | `options.[].label` | +| `prop.{}` | Object values are components | Dynamic keys | +| `prop.{}.sub` | `sub` prop of object values | Nested dynamic | + +### How crawlLayout Works + +```javascript +crawlLayout(object, func, currentPath, extraPath) + +// For each component: +// 1. Call func(object, currentPath) +// 2. If props.children exists, crawl it +// 3. For each path in children_props[namespace][type]: +// - Parse the pattern (handle [], {}) +// - Crawl that path to find nested components +``` + +## Component Resolution + +Components are resolved from `window[namespace][type]`: + +```javascript +// Component packages register on window +window.dash_html_components = { Div, Span, H1, ... }; +window.dash_core_components = { Dropdown, Graph, Input, ... }; + +// Registry.js resolves {type, namespace} → React component +Registry.resolve({type: 'Div', namespace: 'dash_html_components'}) +// → window.dash_html_components.Div +``` + +## Callback Triggering + +Callbacks are triggered by two sources: + +### 1. Component setProps + +When a component calls `setProps`, it triggers callbacks watching those props: + +```javascript +// Component calls setProps +this.props.setProps({ value: newValue }); + +// DashWrapper.tsx handles this: +// 1. dispatch(updateProps(...)) → Updates layout in Redux +// 2. dispatch(notifyObservers(...)) → Finds and queues callbacks +``` + +`notifyObservers` calls `includeObservers` to find callbacks with matching inputs: + +```javascript +// actions/index.js +export function notifyObservers({id, props}) { + return async function (dispatch, getState) { + const {graphs, paths} = getState(); + dispatch( + addRequestedCallbacks(includeObservers(id, props, graphs, paths)) + ); + }; +} +``` + +### 2. Callback Results + +When a callback completes and updates component props, it also triggers dependent callbacks via `includeObservers` in the `executedCallbacks` observer. + +## Callback Processing + +Once callbacks are added to the queue, observers process them through states: + +``` +REQUESTED → PRIORITIZED → EXECUTING → EXECUTED → STORED + ↓ + WATCHED (promises) + ↓ + BLOCKED (waiting on deps) +``` + +### Observer Chain + +1. **requestedCallbacks**: Deduplicates, checks dependencies, moves ready → prioritized +2. **prioritizedCallbacks**: Sorts by priority, executes (max 12 concurrent) +3. **executingCallbacks**: Tracks running callbacks, handles promises +4. **executedCallbacks**: Applies results to layout, triggers dependent callbacks +5. **isLoading**: Tracks loading state for `dcc.Loading` + +## Redux Store + +### Key Slices + +```typescript +{ + layout: { ... }, // Component tree + + layoutHashes: { // Change tracking for memoization + "path": { hash, changedProps } + }, + + paths: { // ID → path mapping + strs: { "my-id": [...path] }, + objs: { "type,index": [...] } // Wildcards + }, + + callbacks: { // Pipeline states + requested, prioritized, blocked, + executing, watched, executed, stored + }, + + graphs: { // Dependency graph + inputMap: { "id": { "prop": [callbacks] } } + }, + + config: { + children_props: { ... }, // From ComponentRegistry + // ... + }, + + isLoading: boolean +} +``` + +### Paths System + +Maps component IDs to their location in layout: + +```typescript +// String IDs +paths.strs["my-dropdown"] = ["layout", "props", "children", 2, "props"] + +// Wildcard IDs (pattern-matching) +paths.objs["type,index"] = [ + { values: ["filter", 0], path: [...] }, + { values: ["filter", 1], path: [...] } +] +``` + +## window.dash_clientside + +The clientside callback API (`utils/clientsideFunctions.ts`): + +```javascript +window.dash_clientside = { + no_update, // Return to skip output + PreventUpdate, // Throw to cancel callback + callback_context, // Current callback info + set_props, // Update props from clientside + clean_url, // URL sanitization + Patch // Partial prop updates +} +``` + +### callback_context + +Available during callback execution: + +```javascript +window.dash_clientside.callback_context = { + triggered: [{ prop_id: "btn.n_clicks", value: 1 }], + triggered_id: "btn", + inputs: { "input.value": "hello" }, + states: { "store.data": {...} } +} +``` + +### Registering Clientside Functions + +```javascript +// In assets/clientside.js +window.dash_clientside = window.dash_clientside || {}; +window.dash_clientside.my_namespace = { + my_function: function(input_value) { + return input_value.toUpperCase(); + } +}; +``` + +### set_props + +Update component props directly from clientside: + +```javascript +// By string ID +window.dash_clientside.set_props('my-component', { value: 'new' }); + +// By pattern-matching ID +window.dash_clientside.set_props({ type: 'input', index: 0 }, { value: 'new' }); +``` + +## window.dash_component_api + +API for components to interact with Dash (`dashApi.ts`): + +```javascript +window.dash_component_api = { + ExternalWrapper, // Render outside main tree + DashContext, // React Context + useDashContext, // Hook for context + getLayout, // Get props by ID/path + stringifyId // Convert wildcard IDs +} +``` + +### getLayout + +Retrieve component props: + +```javascript +const props = window.dash_component_api.getLayout('my-dropdown'); +// → { id: 'my-dropdown', options: [...], value: 'a' } +``` + +### useDashContext + +Hook for components: + +```typescript +const { + componentPath, + isLoading, + useSelector, + useDispatch +} = useDashContext(); +``` + +## Memoization + +DashWrapper uses hash-based memoization. `layoutHashes` tracks which components changed: + +```typescript +layoutHashes["0,props,children"] = { + hash: 42, // Increments on change + changedProps: { value: true } +} +``` + +Components only re-render when their hash changes. + +## Key Files + +| File | Purpose | +|------|---------| +| `DashRenderer.js` | Entry point | +| `AppProvider.react.tsx` | Redux store setup | +| `APIController.react.js` | Fetches layout, hydrates app | +| `wrapper/DashWrapper.tsx` | Component rendering, setProps | +| `actions/utils.js` | `crawlLayout` algorithm | +| `registry.js` | Component resolution | +| `reducers/layout.js` | Layout state | +| `reducers/callbacks.ts` | Callback pipeline | +| `reducers/config.js` | Stores children_props | +| `actions/index.js` | `notifyObservers` | +| `actions/dependencies_ts.ts` | `includeObservers`, callback matching | +| `observers/*.ts` | Callback processing | +| `utils/clientsideFunctions.ts` | Clientside API | +| `dashApi.ts` | Component API | diff --git a/.ai/TESTING.md b/.ai/TESTING.md new file mode 100644 index 0000000000..2e2e562482 --- /dev/null +++ b/.ai/TESTING.md @@ -0,0 +1,496 @@ +# Testing + +Dash includes a pytest/Selenium testing framework for unit and integration tests. Located in `dash/testing/`. + +## Quick Start + +```bash +# Install testing dependencies +pip install -e .[testing] + +## Running tests — CRITICAL RULES + +You must activate the virtual environment before running any tests. + +### Unit tests + +``` +source .venv/bin/activate && python -m pytest "tests/unit/" 2>&1 +``` + +### Integration tests + +Always work with a specific test or test file. NEVER run all integration tests at once. + +``` +source .venv/bin/activate && python -m pytest tests/integration/callbacks/test_basic_callback.py --headless -xvs 2>&1 +``` + +**NEVER truncate test output so aggressively that you cannot see failures.** +`tail -5` or `tail -10` is NEVER acceptable — you will miss the error details and waste time re-running. +Always use `tail -50` at minimum, or omit the tail entirely for short test runs. + +**NEVER use `grep "FAILED"` to filter test output** — it hides the actual error messages. + +When a test fails, you MUST be able to see the assertion error, traceback, and context in a SINGLE run. If your command truncates this, your command is wrong. + +## Fixtures + +The main fixture is `dash_duo` - a composite of server + browser: + +```python +def test_basic_callback(dash_duo): + app = Dash(__name__) + app.layout = html.Div([ + html.Button("Click", id="btn", n_clicks=0), + html.Div(id="output") + ]) + + @app.callback(Output("output", "children"), Input("btn", "n_clicks")) + def update(n): + return f"Clicked {n} times" + + dash_duo.start_server(app) + dash_duo.find_element("#btn").click() + dash_duo.wait_for_text_to_equal("#output", "Clicked 1 times") +``` + +### Available Fixtures + +| Fixture | Description | +|---------|-------------| +| `dash_duo` | Threaded server + browser (default for integration tests) | +| `dash_duo_mp` | Multi-process server + browser | +| `dash_br` | Browser only (no server) | +| `dash_thread_server` | Threaded server only | +| `dash_process_server` | Process-based server only | +| `dashr` | DashR server + browser | +| `dashjl` | Dash.jl server + browser | + +## Browser Methods + +### Element Selection + +```python +dash_duo.find_element("#my-id") # Single element by CSS selector +dash_duo.find_elements(".my-class") # All matching elements +dash_duo.wait_for_element("#loading") # Wait for element to appear +dash_duo.wait_for_element_by_id("output") # Wait by ID +``` + +### Wait Conditions + +```python +# Wait for exact text +dash_duo.wait_for_text_to_equal("#output", "Expected text") + +# Wait for text containing substring +dash_duo.wait_for_contains_text("#output", "partial") + +# Wait for CSS class +dash_duo.wait_for_class_to_equal("#elem", "active") +dash_duo.wait_for_contains_class("#elem", "loading") + +# Wait for CSS property +dash_duo.wait_for_style_to_equal("#elem", "display", "none") + +# Wait for element removal +dash_duo.wait_for_no_elements("#spinner") + +# Custom timeout (default 10s) +dash_duo.wait_for_text_to_equal("#slow", "Done", timeout=30) +``` + +### Interactions + +```python +# Click +dash_duo.find_element("#btn").click() +dash_duo.multiple_click("#btn", clicks=5) + +# Input +elem = dash_duo.find_element("#input") +elem.send_keys("hello") +dash_duo.clear_input("#input") + +# Dropdown +dash_duo.select_dcc_dropdown("#dropdown", value="option1") +dash_duo.select_dcc_dropdown("#dropdown", index=2) + +# Graph interactions +dash_duo.click_at_coord_fractions("#graph", 0.5, 0.5) # Click center +dash_duo.zoom_in_graph_by_ratio("#graph", 0.5, 0.25, 0.5, 0.75) +``` + +### State Inspection + +```python +# Redux state +dash_duo.redux_state_is_loading # True if callbacks running +dash_duo.redux_state_paths # Component paths +dash_duo.redux_state_rqs # Pending requests + +# Storage +dash_duo.get_local_storage("store-id") +dash_duo.get_session_storage("session-id") +dash_duo.clear_storage() + +# DOM access (BeautifulSoup) +dom = dash_duo.dash_outerhtml_dom +assert dom.find(id="my-component") is not None + +# Browser logs (Chrome only) +logs = dash_duo.get_logs() +assert logs == [] # No console errors +``` + +## Application Runners + +Runners manage server lifecycle: + +| Runner | How It Works | Use Case | +|--------|--------------|----------| +| `ThreadedRunner` | Daemon thread | Fast, default | +| `ProcessRunner` | Subprocess + waitress | Production-like | +| `MultiProcessRunner` | Multiprocessing | Multi-worker tests | +| `RRunner` | Rscript subprocess | DashR | +| `JuliaRunner` | Julia subprocess | Dash.jl | + +```python +def test_with_process_server(dash_process_server): + app = Dash(__name__) + app.layout = html.Div("Hello") + + dash_process_server(app) + response = requests.get(dash_process_server.url) + assert response.status_code == 200 +``` + +## Wait Utilities + +For custom wait conditions (`dash/testing/wait.py`): + +```python +from dash.testing.wait import until, until_not + +# Poll until condition is True +until( + lambda: dash_duo.find_element("#status").text == "Ready", + timeout=10, + poll=0.5, + msg="Status never became Ready" +) + +# Poll until condition is False +until_not( + lambda: dash_duo.redux_state_is_loading, + timeout=5 +) +``` + +## Percy Visual Testing + +Percy integration for visual regression testing: + +```python +def test_visual_appearance(dash_duo): + app = Dash(__name__) + app.layout = html.Div([...]) + + dash_duo.start_server(app) + + # Basic snapshot + dash_duo.percy_snapshot("dashboard-initial") + + # Wait for callbacks before snapshot + dash_duo.percy_snapshot( + name="dashboard-loaded", + wait_for_callbacks=True + ) + + # Convert canvas elements to images (for graphs) + dash_duo.percy_snapshot( + name="graph-render", + convert_canvases=True + ) + + # Responsive widths + dash_duo.percy_snapshot( + name="responsive", + widths=[375, 768, 1280] + ) +``` + +Navigate and snapshot in one call: + +```python +dash_duo.visit_and_snapshot( + resource_path="/page2", + hook_id="page2-content", + wait_for_callbacks=True +) +``` + +## CLI Options + +```bash +# Browser selection +pytest --webdriver Chrome # Default +pytest --webdriver Firefox + +# Headless mode +pytest --headless + +# Selenium Grid +pytest --remote --remote-url http://grid:4444/wd/hub + +# Percy +pytest --percy-assets tests/assets +pytest --nopercyfinalize # Don't finalize Percy build + +# Debugging +pytest --pause # Pause with pdb after page load +``` + +## Test Organization + +``` +tests/ # Core Dash tests +├── unit/ # Fast tests, no browser +├── integration/ # Browser-based tests +│ ├── callbacks/ # Callback behavior +│ ├── clientside/ # Clientside callbacks +│ ├── dash/ # Core app features +│ ├── dash_assets/ # Asset loading +│ ├── devtools/ # Dev tools UI +│ ├── multi_page/ # Pages system +│ ├── renderer/ # Frontend rendering +│ └── security/ # Security features +├── async_tests/ # Async callback tests +├── background_callback/ # Background callback tests +├── backend_tests/ # Server-side tests +└── compliance/ # Type checking compliance + └── test_typing.py # pyright/mypy validation + +components/dash-core-components/tests/ # DCC component tests +├── unit/ # Unit tests +└── integration/ # Per-component browser tests + ├── dropdown/ + ├── graph/ + ├── input/ + ├── slider/ + ├── store/ + ├── upload/ + └── ... + +components/dash-html-components/tests/ # HTML component tests +├── test_dash_html_components.py +├── test_div_tabIndex.py +└── test_integration.py + +components/dash-table/tests/ # DataTable tests +├── unit/ # Python unit tests +├── js-unit/ # JavaScript unit tests +├── selenium/ # Browser tests +└── visual/ # Visual regression tests + +dash/dash-renderer/tests/ # Renderer JS tests +├── isAppReady.test.js +└── persistence.test.js +``` + +### Running Component Tests + +```bash +# DCC tests +pytest components/dash-core-components/tests/ + +# Specific DCC component +pytest components/dash-core-components/tests/integration/dropdown/ + +# HTML components +pytest components/dash-html-components/tests/ + +# DataTable +pytest components/dash-table/tests/selenium/ + +# Renderer JS tests +cd dash/dash-renderer && npm test +``` + +## Type Checking Compliance + +The `tests/compliance/test_typing.py` tests validate that Dash code passes static type checkers (pyright, mypy). This ensures type annotations are correct and users get proper IDE support. + +### What It Tests + +1. **Component prop types** - Validates generated TypeScript component types work correctly: + ```python + # Should pass - correct type + TypeScriptComponent(a_string='hello') + + # Should fail - wrong type + TypeScriptComponent(a_string=123) # Expected str, got int + ``` + +2. **Layout types** - Validates layout accepts correct children types: + ```python + # Valid - components, strings, numbers + html.Div([html.H2('Title'), 'text', 123]) + + # Invalid - dict in children + html.Div([{'invalid': 'dict'}]) + ``` + +3. **Callback return types** - Validates callback returns match Output type: + ```python + @callback(Output("out", "children"), Input("in", "value")) + def update() -> html.Div: + return html.Div('Valid') # OK + return [] # Type error + ``` + +### Running Type Checks + +```bash +# Run compliance tests +pytest tests/compliance/ + +# Run pyright directly +pyright dash/ + +# Run mypy directly (Python 3.10+) +mypy dash/ +``` + +### Type Checkers Used + +| Checker | Python Version | Notes | +|---------|---------------|-------| +| pyright | All | Primary checker, always runs | +| mypy | 3.10+ | Runs on Python 3.10 and above | + +## Common Patterns + +### Testing Callbacks + +```python +def test_callback_updates_output(dash_duo): + app = Dash(__name__) + app.layout = html.Div([ + dcc.Input(id="input", value=""), + html.Div(id="output") + ]) + + @app.callback(Output("output", "children"), Input("input", "value")) + def update(value): + return f"You typed: {value}" + + dash_duo.start_server(app) + + input_elem = dash_duo.find_element("#input") + input_elem.send_keys("hello") + + dash_duo.wait_for_text_to_equal("#output", "You typed: hello") + assert dash_duo.get_logs() == [] +``` + +### Testing Loading States + +```python +def test_loading_indicator(dash_duo): + app = Dash(__name__) + app.layout = html.Div([ + html.Button("Load", id="btn"), + dcc.Loading(html.Div(id="output")) + ]) + + @app.callback(Output("output", "children"), Input("btn", "n_clicks")) + def slow_update(n): + time.sleep(1) + return "Loaded" + + dash_duo.start_server(app) + dash_duo.find_element("#btn").click() + + # Verify loading state appears + dash_duo.wait_for_element(".dash-spinner") + + # Then verify it completes + dash_duo.wait_for_text_to_equal("#output", "Loaded") + dash_duo.wait_for_no_elements(".dash-spinner") +``` + +### Testing Background Callbacks + +```python +def test_background_callback(dash_duo, diskcache_manager): + app = Dash(__name__) + app.layout = html.Div([ + html.Button("Start", id="btn"), + html.Div(id="progress"), + html.Div(id="result") + ]) + + @app.callback( + Output("result", "children"), + Input("btn", "n_clicks"), + progress=Output("progress", "children"), + background=True, + manager=diskcache_manager, + ) + def compute(set_progress, n): + for i in range(5): + set_progress(f"{i*20}%") + time.sleep(0.1) + return "Done" + + dash_duo.start_server(app) + dash_duo.find_element("#btn").click() + + dash_duo.wait_for_contains_text("#progress", "%") + dash_duo.wait_for_text_to_equal("#result", "Done") +``` + +### Testing Multi-Page Apps + +```python +def test_page_navigation(dash_duo): + app = Dash(__name__, use_pages=True) + # pages/ directory contains page modules + + dash_duo.start_server(app) + + # Test home page + dash_duo.wait_for_element("#home-content") + + # Navigate to another page + dash_duo.find_element('a[href="/about"]').click() + dash_duo.wait_for_element("#about-content") + + # Check URL updated + assert "/about" in dash_duo.driver.current_url +``` + +## Key Files + +| File | Purpose | +|------|---------| +| `dash/testing/plugin.py` | Pytest plugin, fixture definitions | +| `dash/testing/browser.py` | Browser class with Selenium wrapper | +| `dash/testing/composite.py` | DashComposite (server + browser) | +| `dash/testing/application_runners.py` | Server runners | +| `dash/testing/wait.py` | Wait utilities and conditions | +| `dash/testing/dash_page.py` | Redux state access mixin | +| `dash/testing/errors.py` | Custom exceptions | + +## Errors + +```python +from dash.testing.errors import ( + TestingTimeoutError, # Wait condition timed out + DashAppLoadingError, # App failed to load + ServerCloseError, # Server didn't stop cleanly + BrowserError, # Browser/WebDriver issue +) +``` diff --git a/.ai/TROUBLESHOOTING.md b/.ai/TROUBLESHOOTING.md new file mode 100644 index 0000000000..c8419646a3 --- /dev/null +++ b/.ai/TROUBLESHOOTING.md @@ -0,0 +1,329 @@ +# Troubleshooting + +Common issues and solutions when working with Dash. + +## Callback Errors + +### "Callback error updating [component]" + +**Cause:** Exception raised inside callback function. + +**Solution:** +1. Check the terminal for the full traceback +2. Enable debug mode: `app.run(debug=True)` +3. Add error handling: +```python +@app.callback(Output('out', 'children'), Input('in', 'value'), on_error=lambda e: f"Error: {e}") +def update(value): + ... +``` + +### "A nonexistent object was used in an `Input`..." + +**Cause:** Callback references component ID that doesn't exist in layout. + +**Solutions:** +1. Check for typos in component IDs +2. For dynamic layouts, set `suppress_callback_exceptions=True`: +```python +app = Dash(__name__, suppress_callback_exceptions=True) +``` +3. Use pattern-matching callbacks for dynamic components + +### "Circular dependency detected" + +**Cause:** Callback output is also its own input (directly or indirectly). + +**Solution:** Restructure callbacks to break the cycle. Use `State` instead of `Input` where possible, or split into multiple callbacks. + +### Callback not firing + +**Possible causes:** +1. `prevent_initial_call=True` blocking first execution +2. Input component doesn't exist yet (dynamic layout) +3. Component ID mismatch (check spelling, check dict IDs match exactly) + +**Debug:** Add `print()` at callback start to verify it's being called. + +## Layout Errors + +### "Invalid component type" + +**Cause:** Passing non-component to layout (e.g., raw dict, unsupported type). + +**Solution:** Ensure all layout children are Dash components, strings, or numbers: +```python +# Wrong +html.Div([{'key': 'value'}]) + +# Right +html.Div([html.Span('value')]) +``` + +### Components not rendering + +**Possible causes:** +1. Missing `id` prop (required for callbacks) +2. JavaScript error - check browser console +3. Component library not installed or imported + +**Debug:** Check browser DevTools console for errors. + +## Import Errors + +### "No module named 'dash_core_components'" + +**Cause:** Using old import style. + +**Solution:** Use new unified imports: +```python +# Old (deprecated) +import dash_core_components as dcc +import dash_html_components as html + +# New +from dash import dcc, html +``` + +### "ImportError: cannot import name 'X' from 'dash'" + +**Cause:** Feature not available in installed Dash version. + +**Solution:** Upgrade Dash: +```bash +pip install --upgrade dash +``` + +## Server Errors + +### "Address already in use" + +**Cause:** Port 8050 (or specified port) is occupied. + +**Solutions:** +1. Use different port: `app.run(port=8051)` +2. Kill existing process: `lsof -i :8050` then `kill ` +3. Set via environment: `PORT=8051 python app.py` + +### Hot reload not working + +**Possible causes:** +1. `debug=False` (hot reload requires debug mode) +2. File outside watched directories +3. Syntax error preventing reload + +**Solution:** +```python +app.run( + debug=True, + dev_tools_hot_reload=True, + extra_hot_reload_paths=['./custom_modules/'] +) +``` + +### "Working outside of application context" + +**Cause:** Accessing Flask context outside request (e.g., in background thread). + +**Solution:** Use `flask.current_app` inside callbacks, or pass data explicitly rather than using context. + +## Background Callback Issues + +### Background callback never completes + +**Possible causes:** +1. Manager not configured correctly +2. Celery worker not running (for CeleryManager) +3. Exception in callback (check worker logs) + +**Debug:** Check diskcache directory or Celery worker output for errors. + +### "No such process" errors with DiskcacheManager + +**Cause:** Process terminated unexpectedly. + +**Solution:** Check for exceptions in the callback. Ensure `psutil` is installed. + +### Progress updates not showing + +**Cause:** `set_progress` not being called, or wrong output specified. + +**Solution:** Ensure `progress` parameter matches an Output that exists: +```python +@app.callback( + Output('result', 'children'), + Input('btn', 'n_clicks'), + progress=Output('progress', 'children'), # Must exist in layout + background=True, + manager=manager, +) +def compute(set_progress, n): + set_progress("Working...") # Call this + ... +``` + +## Async Callback Issues + +### "You are trying to use a coroutine without dash[async]" + +**Cause:** Using `async def` callback without async dependencies. + +**Solution:** +```bash +pip install dash[async] +``` + +### Event loop errors in Jupyter + +**Cause:** Conflicting event loops. + +**Solution:** Dash automatically applies `nest_asyncio` in Jupyter. If issues persist: +```python +import nest_asyncio +nest_asyncio.apply() +``` + +## Multi-Page App Issues + +### Pages not discovered + +**Possible causes:** +1. Files don't contain `register_page(__name__)` +2. Files start with `_` or `.` (ignored) +3. Wrong `pages_folder` path + +**Solution:** Ensure each page file has: +```python +from dash import register_page +register_page(__name__) + +layout = ... +``` + +### "Page not found" for registered page + +**Cause:** Path mismatch or routing issue. + +**Debug:** Check `dash.page_registry` to see registered pages and their paths: +```python +from dash import page_registry +print(list(page_registry.values())) +``` + +## Component-Specific Issues + +### Dropdown options not updating + +**Cause:** Options list reference didn't change (same list object). + +**Solution:** Return new list object: +```python +# Wrong - mutating existing list +options.append(new_option) +return options + +# Right - return new list +return options + [new_option] +``` + +### Graph not updating + +**Possible causes:** +1. Returning same figure object (reference equality) +2. Missing `figure` in Output + +**Solution:** Create new figure object: +```python +return go.Figure(data=[...], layout={...}) # New object each time +``` + +### DataTable slow with large data + +**Solutions:** +1. Enable virtualization: `virtualization=True` +2. Use pagination: `page_size=20, page_action='native'` +3. Filter data server-side before sending + +## Testing Issues + +### ChromeDriver version mismatch + +**Error:** "session not created: This version of ChromeDriver only supports Chrome version X" + +**Solution:** Update ChromeDriver to match your Chrome version: +```bash +# Check Chrome version +google-chrome --version + +# Install matching chromedriver +pip install chromedriver-autoinstaller +``` + +### Tests hanging + +**Possible causes:** +1. Callback never completing +2. Element selector not finding element +3. Timeout too short + +**Solution:** Add explicit waits with longer timeout: +```python +dash_duo.wait_for_text_to_equal("#output", "expected", timeout=30) +``` + +### "Element not interactable" + +**Cause:** Element hidden, overlapped, or not yet rendered. + +**Solution:** Wait for element to be visible: +```python +dash_duo.wait_for_element("#button") +element = dash_duo.find_element("#button") +element.click() +``` + +## Build Issues + +### "Component build failed" + +**Possible causes:** +1. Node modules not installed: `npm ci` +2. Syntax error in React component +3. Missing dependencies + +**Solution:** Check build output, ensure `npm ci` was run in component directory. + +### "Module not found" after build + +**Cause:** Python package not installed in editable mode. + +**Solution:** +```bash +pip install -e . +``` + +## Performance Issues + +### App slow to load + +**Solutions:** +1. Use `eager_loading=False` (default) for lazy component loading +2. Minimize assets in `assets/` folder +3. Use `serve_locally=False` to serve from CDN + +### Callbacks slow + +**Solutions:** +1. Use `background=True` for expensive computations +2. Cache results with `@cache.memoize` or similar +3. Use clientside callbacks for simple transformations +4. Reduce data sent to/from server + +### Memory growing over time + +**Possible causes:** +1. Storing data in global variables +2. Background callback results not being cleaned up +3. Large figures being cached + +**Solution:** Use `dcc.Store` for state, set `expire` on background managers, avoid global mutable state. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000000..9b2826a80c --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,42 @@ +# Copilot Instructions + +This file provides guidance to GitHub Copilot when working with this repository. + +For detailed documentation, see the [`.ai/`](../.ai/) directory: +- [Commands](../.ai/COMMANDS.md) - Build, test, lint +- [Architecture](../.ai/ARCHITECTURE.md) - Backend, layout, callbacks +- [Components](../.ai/COMPONENTS.md) - Component system, generation, resources + +## Project Overview + +Dash is a Python framework for building reactive web-based data visualization applications. Built on Plotly.js, React, and Flask. + +## Quick Reference + +```bash +# Setup +pip install -e .[ci,dev,testing,celery,diskcache] && npm ci + +# Build +npm run build # Linux/Mac +npm run first-build # Windows (use Bash) +dash-update-components "dash-core-components" # Single component + +# Test +pytest tests/unit # Unit tests +pytest tests/integration # Integration tests +pytest -k test_name # Specific test + +# Lint +npm run lint # All linters +npm run private::format.black # Auto-format Python +``` + +## Key Files + +- `dash/dash.py` - Main Dash app class, layout, callbacks, routing +- `dash/_callback.py` - `@callback` decorator and execution +- `dash/dependencies.py` - `Input`, `Output`, `State`, wildcards +- `dash/development/base_component.py` - Component base class, `to_plotly_json()` +- `dash/dash-renderer/` - TypeScript/React frontend +- `components/` - Component packages (dcc, html, table) diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..0e97d223fc --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,10 @@ +# AGENTS.md + +For guidance on working with this repository, see the [`.ai/`](.ai/) directory: + +- [COMMANDS.md](.ai/COMMANDS.md) - Build, test, lint +- [ARCHITECTURE.md](.ai/ARCHITECTURE.md) - Backend, callbacks, layout +- [RENDERER.md](.ai/RENDERER.md) - Frontend, React, Redux +- [COMPONENTS.md](.ai/COMPONENTS.md) - Component system +- [TESTING.md](.ai/TESTING.md) - Testing patterns +- [TROUBLESHOOTING.md](.ai/TROUBLESHOOTING.md) - Common issues diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000000..be36e0949e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,48 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Required Reading + +**At the start of each session, read the documentation in `.ai/` before making changes:** + +1. `.ai/COMMANDS.md` - Build, test, lint commands +2. `.ai/ARCHITECTURE.md` - Backend, callbacks, pages, assets, errors, background callbacks, Jupyter, config, stores, async, security +3. `.ai/RENDERER.md` - Frontend, crawlLayout, Redux store, clientside API, component API +4. `.ai/COMPONENTS.md` - Component system, generation, resources +5. `.ai/TESTING.md` - Testing framework, fixtures, patterns, type compliance +6. `.ai/TROUBLESHOOTING.md` - Common errors and solutions + +## Project Overview + +Dash is a Python framework for building reactive web-based data visualization applications. Built on Plotly.js, React, and Flask. + +## Quick Reference + +```bash +# Setup +pip install -e .[ci,dev,testing,celery,diskcache] && npm ci + +# Build +npm run build # Linux/Mac +npm run first-build # Windows (use Bash) +dash-update-components "dash-core-components" # Single component + +# Test +pytest tests/unit # Unit tests +pytest tests/integration # Integration tests +pytest -k test_name # Specific test + +# Lint +npm run lint # All linters +npm run private::format.black # Auto-format Python +``` + +## Key Files + +- `dash/dash.py` - Main Dash app class, layout, callbacks, routing +- `dash/_callback.py` - `@callback` decorator and execution +- `dash/dependencies.py` - `Input`, `Output`, `State`, wildcards +- `dash/development/base_component.py` - Component base class, `to_plotly_json()` +- `dash/dash-renderer/` - TypeScript/React frontend +- `components/` - Component packages (dcc, html, table)