# tidysdmx

## Docs

### API Reference

#### FMR & Schemas
> Fetch and parse SDMX schemas from a Fusion Metadata Registry.

- [fetch_schema](reference/fetch_schema.html): Fetch the schema of a specified artefact from an SDMX registry
- [fetch_dsd_schema](reference/fetch_dsd_schema.html): Fetch a DSD schema from a Fusion Metadata Registry (FMR)
- [parse_artefact_id](reference/parse_artefact_id.html): Parse an artefact identifier into its components: agency, id and version
- [parse_dsd_id](reference/parse_dsd_id.html): Parse a DSD identifier into its components
- [create_schema_from_table](reference/create_schema_from_table.html): Create a DSD, ConceptScheme, and Codelists from a DataFrame

#### Structure Maps
> Build, parse, validate, and write SDMX structure maps.

- [parse_mapping_template_wb](reference/parse_mapping_template_wb.html): Read an Excel mapping template and return all sheets as DataFrames
- [build_structure_map_from_template_wb](reference/build_structure_map_from_template_wb.html): Build a complete StructureMap object by parsing a WB-format Excel template
- [build_fixed_map](reference/build_fixed_map.html): Build a pysdmx FixedValueMap for setting a component to a fixed value
- [build_implicit_component_map](reference/build_implicit_component_map.html): Build a pysdmx ImplicitComponentMap for implicit mapping rules
- [build_date_pattern_map](reference/build_date_pattern_map.html): Build a DatePatternMap object for mapping date patterns between SDMX components
- [build_value_map](reference/build_value_map.html): Create a pysdmx ValueMap object mapping a source value to a target value
- [build_value_map_list](reference/build_value_map_list.html): Build a list of ValueMap objects from a pandas DataFrame, optionally including validity periods
- [build_multi_value_map_list](reference/build_multi_value_map_list.html): Build a list of MultiValueMap objects from a pandas DataFrame
- [build_representation_map](reference/build_representation_map.html): Build a RepresentationMap object from a pandas DataFrame using build_value_map_list
- [build_multi_representation_map](reference/build_multi_representation_map.html): Build a MultiRepresentationMap object from a pandas DataFrame
- [build_single_component_map](reference/build_single_component_map.html): Build a ComponentMap mapping one source component to one target component using a RepresentationMap built from a pandas DataFrame
- [collect_structure_map_artifacts](reference/collect_structure_map_artifacts.html): Collect the StructureMap and all its dependent RepresentationMaps
- [validate_structure_map_references](reference/validate_structure_map_references.html): Validate that all RepresentationMap references are resolved
- [prepare_structure_map_for_upload](reference/prepare_structure_map_for_upload.html): Prepare a StructureMap for upload by collecting all dependencies

#### Mapping
> Apply structure maps to tidy DataFrames.

- [map_structures](reference/map_structures.html): Apply all mapping components from a StructureMap to a DataFrame
- [apply_fixed_value_maps](reference/apply_fixed_value_maps.html): Apply FixedValueMap rules to a DataFrame
- [apply_implicit_component_maps](reference/apply_implicit_component_maps.html): Apply ImplicitComponentMap rules to a DataFrame
- [apply_multi_component_map](reference/apply_multi_component_map.html): Apply a single MultiComponentMap with regex support, preserving rule order
- [map_to_sdmx](reference/map_to_sdmx.html): Map DataFrame columns to SDMX values using a lookup mapping
- [transform_source_to_target](reference/transform_source_to_target.html): Transform a raw DataFrame into the format defined by a components map

#### Standardisation
> Prepare a mapped DataFrame for SDMX upload.

- [standardize_output](reference/standardize_output.html): Standardize the output DataFrame by adding SDMX reference columns
- [standardize_sdmx](reference/standardize_sdmx.html): Standardize a DataFrame by applying column and value transformations
- [standardize_data_for_upload](reference/standardize_data_for_upload.html): Standardize a DataFrame for SDMX upload
- [standardize_indicator_id](reference/standardize_indicator_id.html): Fix the INDICATOR column to be uppercase and prefixed with dataset ID
- [sanitize_variable](reference/sanitize_variable.html): Sanitize a raw string value into a valid SDMX code ID
- [add_sdmx_reference_cols](reference/add_sdmx_reference_cols.html): Add SDMX reference columns to a DataFrame

#### Validation
> Validate datasets against schemas and codelists.

- [validate_dataset_local](reference/validate_dataset_local.html): Validate that a DataFrame is SDMX compliant and return a DataFrame of errors
- [validate_columns](reference/validate_columns.html): Validate that all DataFrame columns are valid components or SDMX references
- [validate_mandatory_columns](reference/validate_mandatory_columns.html): Validate that all mandatory columns are present in the DataFrame
- [validate_codelist_ids](reference/validate_codelist_ids.html): Validate that all values in coded columns are within the allowed codelist IDs
- [validate_duplicates](reference/validate_duplicates.html): Validate that there are no duplicate rows for a given set of key columns
- [validate_no_missing_values](reference/validate_no_missing_values.html): Validate that there are no missing values in mandatory columns

#### Tidy Raw
> Filter and shape raw inputs.

- [filter_tidy_raw](reference/filter_tidy_raw.html): Filter an SDMX DataFrame by removing rows that violate codelist constraints
- [filter_rows](reference/filter_rows.html): Filter out rows where values are not in the allowed codelist

#### Utilities
> Helpers for codelists, components, Excel templates, and XML.

- [extract_validation_info](reference/extract_validation_info.html): Extract validation information from a given schema
- [get_codelist_ids](reference/get_codelist_ids.html): Retrieve all codelist IDs for given coded components
- [extract_component_ids](reference/extract_component_ids.html): Retrieve all component IDs from a given pysdmx Schema
- [create_mapping_rules](reference/create_mapping_rules.html): Create Excel hyperlink formulas for components with representation maps
- [build_excel_workbook](reference/build_excel_workbook.html): Build a Workbook with component mapping and representation map sheets
- [write_excel_mapping_template](reference/write_excel_mapping_template.html): Generate an Excel mapping template with component and representation tabs
- [read_mapping](reference/read_mapping.html): Read a JSON mapping file and parse its content into DataFrames
- [fix_sdmx_xml_datatype_tags](reference/fix_sdmx_xml_datatype_tags.html): Fix incorrect SourceCodelist/TargetCodelist tags in SDMX-ML
- [gen_urn](reference/gen_urn.html): Generate a full SDMX URN for any maintainable artefact

#### QA
> Quality-assurance helpers.

- [qa_coerce_numeric](reference/qa_coerce_numeric.html): Coerce specified columns to numeric, removing rows with invalid values
- [qa_remove_duplicates](reference/qa_remove_duplicates.html): Remove duplicate rows from a DataFrame

#### Kedro Integration
> Kedro pipeline node wrappers.

- [kd_read_mappings](reference/kd_read_mappings.html): Fetch multiple mappings from different files
- [kd_standardize_sdmx](reference/kd_standardize_sdmx.html): Standardize a partitioned dataset into SDMX format
- [kd_validate_dataset_local](reference/kd_validate_dataset_local.html): Validate a single DataFrame for SDMX compliance
- [kd_validate_datasets_local](reference/kd_validate_datasets_local.html): Validate multiple datasets for SDMX compliance

#### Lookups
> Vectorised lookup helpers.

- [vectorized_lookup_ordered_v1](reference/vectorized_lookup_ordered_v1.html): Apply ordered regex matching to a Pandas Series
- [vectorized_lookup_ordered_v2](reference/vectorized_lookup_ordered_v2.html): Apply ordered matching (regex or exact) to a Pandas Series