# tidysdmx # Installation ``` bash pip install tidysdmx ``` # API overview ## FMR & Schemas Fetch and parse SDMX schemas from a Fusion Metadata Registry. - [fetch_schema](reference/fetch_schema.html#tidysdmx.fetch_schema): Fetch the schema of a specified artefact from an SDMX registry - [fetch_dsd_schema](reference/fetch_dsd_schema.html#tidysdmx.fetch_dsd_schema): Fetch a DSD schema from a Fusion Metadata Registry (FMR) - [parse_artefact_id](reference/parse_artefact_id.html#tidysdmx.parse_artefact_id): Parse an artefact identifier into its components: agency, id and version - [parse_dsd_id](reference/parse_dsd_id.html#tidysdmx.parse_dsd_id): Parse a DSD identifier into its components - [create_schema_from_table](reference/create_schema_from_table.html#tidysdmx.create_schema_from_table): Create a DSD, ConceptScheme, and Codelists from a DataFrame ## Structure Maps Build, parse, validate, and write SDMX structure maps. - [parse_mapping_template_wb](reference/parse_mapping_template_wb.html#tidysdmx.parse_mapping_template_wb): Read an Excel mapping template and return all sheets as DataFrames - [build_structure_map_from_template_wb](reference/build_structure_map_from_template_wb.html#tidysdmx.build_structure_map_from_template_wb): Build a complete StructureMap object by parsing a WB-format Excel template - [build_fixed_map](reference/build_fixed_map.html#tidysdmx.build_fixed_map): Build a pysdmx FixedValueMap for setting a component to a fixed value - [build_implicit_component_map](reference/build_implicit_component_map.html#tidysdmx.build_implicit_component_map): Build a pysdmx ImplicitComponentMap for implicit mapping rules - [build_date_pattern_map](reference/build_date_pattern_map.html#tidysdmx.build_date_pattern_map): Build a DatePatternMap object for mapping date patterns between SDMX components - [build_value_map](reference/build_value_map.html#tidysdmx.build_value_map): Create a pysdmx ValueMap object mapping a source value to a target value - [build_value_map_list](reference/build_value_map_list.html#tidysdmx.build_value_map_list): Build a list of ValueMap objects from a pandas DataFrame, optionally including validity periods - [build_multi_value_map_list](reference/build_multi_value_map_list.html#tidysdmx.build_multi_value_map_list): Build a list of MultiValueMap objects from a pandas DataFrame - [build_representation_map](reference/build_representation_map.html#tidysdmx.build_representation_map): Build a RepresentationMap object from a pandas DataFrame using build_value_map_list - [build_multi_representation_map](reference/build_multi_representation_map.html#tidysdmx.build_multi_representation_map): Build a MultiRepresentationMap object from a pandas DataFrame - [build_single_component_map](reference/build_single_component_map.html#tidysdmx.build_single_component_map): Build a ComponentMap mapping one source component to one target component using a RepresentationMap built from a pandas DataFrame - [collect_structure_map_artifacts](reference/collect_structure_map_artifacts.html#tidysdmx.collect_structure_map_artifacts): Collect the StructureMap and all its dependent RepresentationMaps - [validate_structure_map_references](reference/validate_structure_map_references.html#tidysdmx.validate_structure_map_references): Validate that all RepresentationMap references are resolved - [prepare_structure_map_for_upload](reference/prepare_structure_map_for_upload.html#tidysdmx.prepare_structure_map_for_upload): Prepare a StructureMap for upload by collecting all dependencies ## Mapping Apply structure maps to tidy DataFrames. - [map_structures](reference/map_structures.html#tidysdmx.map_structures): Apply all mapping components from a StructureMap to a DataFrame - [apply_fixed_value_maps](reference/apply_fixed_value_maps.html#tidysdmx.apply_fixed_value_maps): Apply FixedValueMap rules to a DataFrame - [apply_implicit_component_maps](reference/apply_implicit_component_maps.html#tidysdmx.apply_implicit_component_maps): Apply ImplicitComponentMap rules to a DataFrame - [apply_multi_component_map](reference/apply_multi_component_map.html#tidysdmx.apply_multi_component_map): Apply a single MultiComponentMap with regex support, preserving rule order - [map_to_sdmx](reference/map_to_sdmx.html#tidysdmx.map_to_sdmx): Map DataFrame columns to SDMX values using a lookup mapping - [transform_source_to_target](reference/transform_source_to_target.html#tidysdmx.transform_source_to_target): Transform a raw DataFrame into the format defined by a components map ## Standardisation Prepare a mapped DataFrame for SDMX upload. - [standardize_output](reference/standardize_output.html#tidysdmx.standardize_output): Standardize the output DataFrame by adding SDMX reference columns - [standardize_sdmx](reference/standardize_sdmx.html#tidysdmx.standardize_sdmx): Standardize a DataFrame by applying column and value transformations - [standardize_data_for_upload](reference/standardize_data_for_upload.html#tidysdmx.standardize_data_for_upload): Standardize a DataFrame for SDMX upload - [standardize_indicator_id](reference/standardize_indicator_id.html#tidysdmx.standardize_indicator_id): Fix the INDICATOR column to be uppercase and prefixed with dataset ID - [sanitize_variable](reference/sanitize_variable.html#tidysdmx.sanitize_variable): Sanitize a raw string value into a valid SDMX code ID - [add_sdmx_reference_cols](reference/add_sdmx_reference_cols.html#tidysdmx.add_sdmx_reference_cols): Add SDMX reference columns to a DataFrame ## Validation Validate datasets against schemas and codelists. - [validate_dataset_local](reference/validate_dataset_local.html#tidysdmx.validate_dataset_local): Validate that a DataFrame is SDMX compliant and return a DataFrame of errors - [validate_columns](reference/validate_columns.html#tidysdmx.validate_columns): Validate that all DataFrame columns are valid components or SDMX references - [validate_mandatory_columns](reference/validate_mandatory_columns.html#tidysdmx.validate_mandatory_columns): Validate that all mandatory columns are present in the DataFrame - [validate_codelist_ids](reference/validate_codelist_ids.html#tidysdmx.validate_codelist_ids): Validate that all values in coded columns are within the allowed codelist IDs - [validate_duplicates](reference/validate_duplicates.html#tidysdmx.validate_duplicates): Validate that there are no duplicate rows for a given set of key columns - [validate_no_missing_values](reference/validate_no_missing_values.html#tidysdmx.validate_no_missing_values): Validate that there are no missing values in mandatory columns ## Tidy Raw Filter and shape raw inputs. - [filter_tidy_raw](reference/filter_tidy_raw.html#tidysdmx.filter_tidy_raw): Filter an SDMX DataFrame by removing rows that violate codelist constraints - [filter_rows](reference/filter_rows.html#tidysdmx.filter_rows): Filter out rows where values are not in the allowed codelist ## Utilities Helpers for codelists, components, Excel templates, and XML. - [extract_validation_info](reference/extract_validation_info.html#tidysdmx.extract_validation_info): Extract validation information from a given schema - [get_codelist_ids](reference/get_codelist_ids.html#tidysdmx.get_codelist_ids): Retrieve all codelist IDs for given coded components - [extract_component_ids](reference/extract_component_ids.html#tidysdmx.extract_component_ids): Retrieve all component IDs from a given pysdmx Schema - [create_mapping_rules](reference/create_mapping_rules.html#tidysdmx.create_mapping_rules): Create Excel hyperlink formulas for components with representation maps - [build_excel_workbook](reference/build_excel_workbook.html#tidysdmx.build_excel_workbook): Build a Workbook with component mapping and representation map sheets - [write_excel_mapping_template](reference/write_excel_mapping_template.html#tidysdmx.write_excel_mapping_template): Generate an Excel mapping template with component and representation tabs - [read_mapping](reference/read_mapping.html#tidysdmx.read_mapping): Read a JSON mapping file and parse its content into DataFrames - [fix_sdmx_xml_datatype_tags](reference/fix_sdmx_xml_datatype_tags.html#tidysdmx.fix_sdmx_xml_datatype_tags): Fix incorrect SourceCodelist/TargetCodelist tags in SDMX-ML - [gen_urn](reference/gen_urn.html#tidysdmx.gen_urn): Generate a full SDMX URN for any maintainable artefact ## QA Quality-assurance helpers. - [qa_coerce_numeric](reference/qa_coerce_numeric.html#tidysdmx.qa_coerce_numeric): Coerce specified columns to numeric, removing rows with invalid values - [qa_remove_duplicates](reference/qa_remove_duplicates.html#tidysdmx.qa_remove_duplicates): Remove duplicate rows from a DataFrame ## Kedro Integration Kedro pipeline node wrappers. - [kd_read_mappings](reference/kd_read_mappings.html#tidysdmx.kd_read_mappings): Fetch multiple mappings from different files - [kd_standardize_sdmx](reference/kd_standardize_sdmx.html#tidysdmx.kd_standardize_sdmx): Standardize a partitioned dataset into SDMX format - [kd_validate_dataset_local](reference/kd_validate_dataset_local.html#tidysdmx.kd_validate_dataset_local): Validate a single DataFrame for SDMX compliance - [kd_validate_datasets_local](reference/kd_validate_datasets_local.html#tidysdmx.kd_validate_datasets_local): Validate multiple datasets for SDMX compliance ## Lookups Vectorised lookup helpers. - [vectorized_lookup_ordered_v1](reference/vectorized_lookup_ordered_v1.html#tidysdmx.vectorized_lookup_ordered_v1): Apply ordered regex matching to a Pandas Series - [vectorized_lookup_ordered_v2](reference/vectorized_lookup_ordered_v2.html#tidysdmx.vectorized_lookup_ordered_v2): Apply ordered matching (regex or exact) to a Pandas Series # Resources - [llms.txt](llms.txt) -- Indexed API reference for LLMs - [llms-full.txt](llms-full.txt) -- Comprehensive documentation for LLMs