Fiftyone MCP Server

Get the color scheme configuration for a dataset. Returns color_by mode, color pool, and per-field color settings.

Gets information about the current FiftyOne App context, including whether it's active and what dataset is loaded.

Get the full field schema for a dataset with complete type information. Returns field_name -> {type, subfield, embedded_doc_type, description, required, read_only} for every sample field and every frame field (for video datasets). More detailed than load_dataset, which only returns field names. This is an MCP tool, not FiftyOne's Dataset.get_field_schema() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.

Get the input schema for a specific operator. Returns a filtered schema by default (UI-only metadata stripped, choices capped at 20). Pass verbose=true only if you need the raw schema. Schemas are context-aware and change based on the current dataset/view/selection. Use this AFTER list_operators to understand what parameters an operator accepts. For dynamic operators, pass 'params' with partial values to resolve fields that depend on earlier selections. Requires either an execution context or dataset_name.

Gets detailed information about a specific FiftyOne plugin including its operators, version, and metadata.

Get the sidebar group configuration for a dataset. Returns the list of groups with their field paths and expanded state.

Get the raw values of a field for all samples in a dataset. Returns a list of values in sample order. Useful for reading custom field data, scores, or metadata. Capped at 10,000 samples by default to avoid large responses. This is an MCP tool, not FiftyOne's Dataset.values() method -- pass dataset_name and field as separate keyword arguments, not field_or_expr.

Compute a histogram of values for a numeric field. Returns bin counts, bin edges, and the count of values outside the range. Useful for visualizing distributions of confidence scores, object sizes, etc. This is an MCP tool, not FiftyOne's Dataset.histogram_values() method -- pass dataset_name and field as separate keyword arguments, not field_or_expr.

Launches the FiftyOne App server. Required for executing delegated operators that need background execution (e.g., brain operators like find_near_duplicates).

List brain method run keys for a dataset. Brain runs include: similarity indexes (used for nearest-neighbour search), visualization embeddings (UMAP, t-SNE), uniqueness scores, and hardness scores. Filter by type (e.g. 'similarity', 'visualization') or method (e.g. 'umap', 'resnet50'). Returns a list of run keys you can pass to get_brain_info or load_brain_results. This is an MCP tool, not FiftyOne's Dataset.list_brain_runs() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.

List all available FiftyOne datasets with metadata. Capped at 50 by default; check 'total' in the response to see if more exist.

List delegated (background) operations and their status. Use this to check on operations that were queued via execute_operator or execute_pipeline with delegate=true. Filter by run state, dataset, or operator URI.

List evaluation run keys for a dataset. Evaluation runs include model performance assessments (detection, classification, segmentation). Filter by type (e.g. 'detection', 'classification') or method (e.g. 'coco', 'open-images'). Returns a list of eval keys you can pass to get_evaluation_info or load_evaluation_results. This is an MCP tool, not FiftyOne's Dataset.list_evaluations() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.

List model evaluation scenarios for a dataset. Scenarios are configurations stored by the ME panel that define how an evaluation is sliced for comparative analysis (by field value, saved view, or custom code). Requires the @voxel51/model_evaluation plugin and at least one scenario to have been configured.

List all panels currently open in the FiftyOne App. Returns each panel's id (use this with set_panel_state, patch_panel_state, or close_panel), name (e.g. 'Samples', 'Histograms', 'Embeddings', 'Map'), and pinned status. Use this before calling set_panel_state to discover the panel_id you need.

Discover all available FiftyOne operators (80+ built-in operators from plugins). Returns operators from installed plugins including: @voxel51/operators (50+ core operators like tag_samples, clone_samples), @voxel51/brain (similarity, duplicates, visualization), @voxel51/utils (create_dataset, delete_dataset, clone_dataset), @voxel51/io (import/export), @voxel51/evaluation, @voxel51/annotation, @voxel51/zoo. Use this FIRST to discover what operators are available before executing them. Returns a slim summary (uri, label, description) by default; pass verbose=true only if you need full operator …

Lists available FiftyOne plugins and their operators. Plugins extend functionality by providing additional operators. Key plugins: @voxel51/brain (16 operators for similarity/duplicates/visualization), @voxel51/utils (12 operators for dataset CRUD), @voxel51/io (5 operators for import/export), @voxel51/evaluation (5 operators), @voxel51/annotation (6 operators), @voxel51/zoo (2 operators). Use this to discover what plugins are installed and what operators they provide. Capped at 50 by default; check 'total' in the response to see if more exist.

Load a FiftyOne dataset by name and return basic information

Compute the mean (average) of a numeric field across all samples. Returns a single float. Useful for average confidence, uniqueness scores, etc. This is an MCP tool, not FiftyOne's Dataset.mean() method -- pass dataset_name and field as separate keyword arguments, not field_or_expr.

Reset the App color scheme to the dataset or global config defaults for the current session. Session-only — does not persist to the database. Use set_color_scheme to persist changes.

Control which fields are visible in the App sidebar. Persists to the database. Use exclude=false (default) to show only the listed paths, or exclude=true to hide them.

Set the color scheme for a dataset. Persists to the database. Supports setting color_by mode, a shared color pool, and per-field color overrides.

Set the sidebar filter state in the App. Filters are applied on top of the current view and control what samples are shown in the grid. This is NOT the same as adding view stages — it sets the Recoil fos.filters UI state directly. Pass a {field_path: FilterState} dict. Label field filters support: values (list), exclude, isMatching, onlyMatch. Numeric field filters support: range ([min, max]), none, nan. Boolean filters support: true, false, none.

Set the live color scheme for the current App session. Updates the in-memory Recoil color state immediately — session-only, lost on page reload. Use set_color_scheme to persist to the database. Supports a color-blind-friendly preset via color_pool_preset='color-blind-friendly'.

Replace the sidebar group configuration for a dataset. Persists to the database. Each group has a name, a list of field paths, and an optional expanded flag.

Sets the workspace (spaces layout) in the App by loading a saved workspace by name.

Bulk-assign values to a field across multiple samples. Accepts either a list of values (one per sample in dataset order) or a {sample_id: value} dict for explicit assignment. Use this when you need to write computed scores, labels, or metadata back to the dataset. Note: the edit_field_values operator does value remapping (old->new), not bulk assignment. This is an MCP tool, not FiftyOne's Dataset.set_values() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.

Sets a filtered view in the FiftyOne App. Supports dynamic stage-based views (Match, FilterLabels, SortBy, Limit, etc. with F() expressions), saved views, or simple convenience filters. The view updates immediately in the App UI. Requires an execution context (call via MCPToolExecutor).

Compute the standard deviation of a numeric field across all samples. Returns a single float. This is an MCP tool, not FiftyOne's Dataset.std() method -- pass dataset_name and field as separate keyword arguments, not field_or_expr.

Compute the sum of a numeric field across all samples. Returns a single number. This is an MCP tool, not FiftyOne's Dataset.sum() method -- pass dataset_name and field as separate keyword arguments, not field_or_expr.

Add tags to samples in a dataset. Optionally filter which samples to tag using view_stages or sample_ids. Returns the count of samples that were tagged. Tags can be used to mark subsets for review, training splits, or any custom categorization. This is an MCP tool, not FiftyOne's Dataset.tag_samples() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.

Remove tags from samples in a dataset. Optionally filter which samples to untag using view_stages or sample_ids. Returns the count of samples that were processed. This is an MCP tool, not FiftyOne's Dataset.untag_samples() method -- it still requires dataset_name as an explicit argument, unlike the bound SDK method which needs none.