Datasource

While working with Firefly.ai’s API, a Datasource represents the raw CSV files. This data can be used either for model training purposes or for running batch predictions once they have been analyzed.

‘Datasource’ API includes creating a Datasource from an uploaded CSV file, querying existing Datasources (Get, List, Preview and Delete) and getting Datasource metadata (e.g. feature types and type insights).

class fireflyai.resources.datasource.Datasource

classmethod create(filename: str, na_values: List[str] = None, wait: bool = False, skip_if_exists: bool = False, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Uploads a file to the server to creates a new Datasource.

Parameters:

• filename (str) – File to be uploaded.
• na_values (Optional[List[str]]) – List of user specific Null values.
• wait (Optional[bool]) – Should the call be synchronous or not.
• skip_if_exists (Optional[bool]) – Check if a Datasource with same name exists and skip if true.
• api_key (Optional[str]) – Explicit api_key, not required if fireflyai.authenticate was run prior.

Returns:

Datasource ID, if successful and wait=False or Datasource if successful and wait=True; raises FireflyError otherwise.

Return type:

FireflyResponse

classmethod create_from_dataframe(df, data_source_name: str, na_values: List[str] = None, wait: bool = False, skip_if_exists: bool = False, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Creates a Datasource from pandas DataFrame.

Parameters:

• df (pandas.DataFrame) – DataFrame object to upload to server.
• data_source_name (str) – Name of the Datasource.
• na_values (Optional[List[str]]) – List of user specific Null values.
• wait (Optional[bool]) – Should the call be synchronous or not.
• skip_if_exists (Optional[bool]) – Check if a Datasource with same name exists and skip if true.
• api_key (Optional[str]) – Explicit api_key, not required, if fireflyai.authenticate() was run prior.

Returns:

Datasource ID, if successful and wait=False or Datasource if successful and wait=True; raises FireflyError otherwise.

Return type:

FireflyResponse

classmethod delete(id: int, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Deletes a specific Datasource.

Parameters:

• id (int) – Datasource ID.
• api_key (Optional[str]) – Explicit api_key, not required, if fireflyai.authenticate() was run prior.

Returns:

“true” if deleted successfuly, raises FireflyClientError otherwise.

Return type:

FireflyResponse

classmethod get(id: int, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Gets information on a specific Datasource.

Information includes the state of the Datasource and other attributes.

Parameters:

• id (int) – Datasource ID.
• api_key (Optional[str]) – Explicit api_key, not required if fireflyai.authenticate was run prior.

Returns:

Information about the Datasource.

Return type:

FireflyResponse

classmethod get_base_types(id: int, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Gets base types of features for a specific Datasource.

Parameters:

• id (int) – Datasource ID.
• api_key (Optional[str]) – Explicit api_key, not required, if fireflyai.authenticate() was run prior.

Returns:

Contains mapping of feature names to base types.

Return type:

FireflyResponse

classmethod get_by_name(name: str, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Gets information on a specific Datasource identified by its name.

Information includes the state of the Datasource and other attributes. Similar to calling fireflyai.Datasource.list(filters_={‘name’: [NAME]}).

Parameters:

• name (str) – Datasource name.
• api_key (Optional[str]) – Explicit api_key, not required if fireflyai.authenticate was run prior.

Returns:

Information about the Datasource.

Return type:

FireflyResponse

classmethod get_feature_types(id: int, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Gets feature types of features for a specific Datasource.

Parameters:

• id (int) – Datasource ID.
• api_key (Optional[str]) – Explicit api_key, not required if fireflyai.authenticate was run prior.

Returns:

Contains mapping of feature names to feature types.

Return type:

FireflyResponse

classmethod get_type_warnings(id: int, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Gets type warning of features for a specific Datasource.

Parameters:

• id (int) – Datasource ID.
• api_key (Optional[str]) – Explicit api_key, not required if fireflyai.authenticate was run prior.

Returns:

Contains mapping of feature names to a list of type warnings (can be empty).

Return type:

FireflyResponse

classmethod list(search_term: str = None, page: int = None, page_size: int = None, sort: Dict[KT, VT] = None, filter_: Dict[KT, VT] = None, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Lists the existing Datasources - supports filtering, sorting and pagination.

Parameters:

• search_term (Optional[str]) – Return only records that contain the search_term in any field.
• page (Optional[int]) – For pagination, which page to return.
• page_size (Optional[int]) – For pagination, how many records will appear in a single page.
• sort (Optional[Dict[str, Union[str, int]]]) – Dictionary of rules to sort the results by.
• filter (Optional[Dict[str, Union[str, int]]]) – Dictionary of rules to filter the results by.
• api_key (Optional[str]) – Explicit api_key, not required, if fireflyai.authenticate() was run prior.

Returns:

Datasources are represented as nested dictionaries under hits.

Return type:

FireflyResponse

classmethod prepare_data(datasource_id: int, dataset_name: str, target: str, problem_type: fireflyai.enums.ProblemType, header: bool = True, na_values: List[str] = None, retype_columns: Dict[str, fireflyai.enums.FeatureType] = None, rename_columns: List[str] = None, datetime_format: str = None, time_axis: str = None, block_id: List[str] = None, sample_id: List[str] = None, subdataset_id: List[str] = None, sample_weight: List[str] = None, not_used: List[str] = None, hidden: List[str] = False, wait: bool = False, skip_if_exists: bool = False, api_key: str = None) → fireflyai.firefly_response.FireflyResponse

Creates and prepares a Dataset.

While creating a Dataset, the feature roles are labeled and the feature types can be set by the user. Data analysis is done in order to optimize model training and search process.

Parameters:

• datasource_id (int) – Datasource ID.
• dataset_name (str) – The name of the Dataset.
• target (str) – The feature name of the target if header=True, otherwise the column index. #TODO
• problem_type (ProblemType) – The problem type.
• header (bool) – Does the file include a header row or not.
• na_values (Optional[List[str]]) – List of user specific Null values.
• retype_columns (Dict[str, FeatureType]) – Change the types of certain columns.
• rename_columns (Optional[List[str]]) – ??? #TODO
• datetime_format (Optional[str]) – The datetime format used in the data.
• time_axis (Optional[str]) – In timeseries problems, the feature that is the time axis.
• block_id (Optional[List[str]]) – To avoid data leakage, data can be split into blocks. Rows with the same block_id, must all be in the train set or the test set. Requires at least 50 unique values in the data.
• sample_id (Optional[List[str]]) – Row identifier.
• subdataset_id (Optional[List[str]]) – Features which specify a subdataset ID in the data.
• sample_weight (Optional[List[str]]) – ??? #TODO
• not_used (Optional[List[str]]) – List of features to ignore.
• hidden (Optional[List[str]]) – ??? #TODO
• wait (Optional[bool]) – Should the call be synchronous or not.
• skip_if_exists (Optional[bool]) – Check if a Dataset with same name exists and skip if true.
• api_key (Optional[str]) – Explicit api_key, not required, if fireflyai.authenticate() was run prior.

Returns:

Dataset ID, if successful and wait=False or Dataset if successful and wait=True; raises FireflyError otherwise.

Return type:

FireflyResponse