ephemeris package

Submodules

ephemeris.common_parser module

class ephemeris.common_parser.ArgumentDefaultsHideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: HideUnderscoresHelpFormatter, ArgumentDefaultsHelpFormatter

class ephemeris.common_parser.HideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: HelpFormatter

add_arguments(actions)

class ephemeris.common_parser.RawDescriptionHideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: HideUnderscoresHelpFormatter, RawDescriptionHelpFormatter

ephemeris.common_parser.add_log_file_argument(parser_or_group)

ephemeris.common_parser.add_verbosity_argument(parser_or_group)

ephemeris.common_parser.get_common_args(login_required=True, log_file=False)

ephemeris.ephemeris_log module

class ephemeris.ephemeris_log.ProgressConsoleHandler(stream=None)

Bases: StreamHandler

A handler class which allows the cursor to stay on one line for selected messages

emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

on_same_line = False

ephemeris.ephemeris_log.disable_external_library_logging()

ephemeris.ephemeris_log.setup_global_logger(name, log_file=None)

ephemeris.generate_tool_list_from_ga_workflow_files module

Tool to generate tools from workflows

ephemeris.generate_tool_list_from_ga_workflow_files.extract_tool_shed_repositories_from_workflow_dict(workflow_dictionary)

ephemeris.generate_tool_list_from_ga_workflow_files.generate_repo_list_from_workflow(workflow_files: Iterable[str], panel_label: str) → List[InstallRepoDict]

ephemeris.generate_tool_list_from_ga_workflow_files.generate_tool_list_from_workflow(workflow_files: Iterable[str], panel_label: str, output_file: str)

Return type:

object

ephemeris.generate_tool_list_from_ga_workflow_files.get_workflow_dictionary(json_file)

ephemeris.generate_tool_list_from_ga_workflow_files.main(argv=None)

ephemeris.generate_tool_list_from_ga_workflow_files.print_yaml_tool_list(tool_dictionary, output_file)

ephemeris.generate_tool_list_from_ga_workflow_files.reduce_tool_list(tool_list: List[InstallRepoDict]) → List[InstallRepoDict]

ephemeris.generate_tool_list_from_ga_workflow_files.translate_workflow_dictionary_to_tool_list(workflow_dictionary, panel_label: str) → List[InstallRepoDict]

ephemeris.get_tool_list_from_galaxy module

Tool to extract a tool list from galaxy.

class ephemeris.get_tool_list_from_galaxy.GiToToolYaml(gi, include_tool_panel_section_id=False, skip_tool_panel_section_name=True, skip_changeset_revision=False, get_data_managers=False, get_all_tools=False)

Bases: object

filter_section_name_or_id_or_changeset(repository_list)

property installed_tool_list

gets a tool list from the toolclient :return:

property repository_list

Toolbox elements returned by api/tools may be of class ToolSection or Tool. Parse these accordingly to get a list of repositories.

property tool_list

property toolbox

Gets the toolbox elements from <galaxy_url>/api/tools

write_to_yaml(output_file)

ephemeris.get_tool_list_from_galaxy.check_galaxy_version(gi)

ephemeris.get_tool_list_from_galaxy.get_repo_from_tool(tool)

Get the minimum items required for re-installing a (list of) tools

ephemeris.get_tool_list_from_galaxy.get_tool_panel(gi)

ephemeris.get_tool_list_from_galaxy.get_tools(gi)

ephemeris.get_tool_list_from_galaxy.main(argv=None)

ephemeris.get_tool_list_from_galaxy.merge_repository_changeset_revisions(repository_list)

Each installed changeset revision of a tool is listed individually. Merge revisions of the same tool into a list.

ephemeris.get_tool_list_from_galaxy.the_same_repository(repo_1_info, repo_2_info, check_revision=True)

Given two dicts containing info about repositories, determine if they are the same repository. Each of the dicts must have the following keys: changeset_revisions`( if check revisions is true), `name, owner, and (either tool_shed or tool_shed_url).

ephemeris.get_tool_list_from_galaxy.tools_for_repository(gi, repository, all_tools=False)

ephemeris.get_tool_list_from_galaxy.walk_tools(tool_panel, f)

ephemeris.install_tool_deps module

Tool to install tool dependencies on a Galaxy instance.

ephemeris.install_tool_deps.main(argv=None)

This script uses bioblend to trigger dependencies installations for the provided tools

ephemeris.run_data_managers module

Run-data-managers is a tool for provisioning data on a galaxy instance.

Run-data-managers has the ability to run multiple data managers that are interdependent. When a reference genome is needed for bwa-mem for example, Run-data-managers can first run a data manager to fetch the fasta file and run another data manager that indexes the fasta file for bwa-mem. This functionality depends on the “watch_tool_data_dir” setting in galaxy.ini to be True. Also, if a new data manager is installed, galaxy needs to be restarted in order for it’s tool_data_dir to be watched.

Run-data-managers needs a yaml that specifies what data managers are run and with which settings. Example files can be found here, here, and here.

By default run-data-managers skips entries in the yaml file that have already been run. It checks it in the following way:

• If the data manager has input variables “name” or “sequence_name” it will check if the “name” column in the data table already has this entry. “name” will take precedence over “sequence_name”.

• If the data manager has input variables “value”, “sequence_id” or ‘dbkey’ it will check if the “value” column in the data table already has this entry. Value takes precedence over sequence_id which takes precedence over dbkey.

• If none of the above input variables are specified the data manager will always run.

class ephemeris.run_data_managers.DataManagers(galaxy_instance: GalaxyInstance, configuration)

Bases: object

data_table_entry_exists(data_table_name, entry, column='value')

Checks whether an entry exists in the a specified column in the data_table.

dm_is_fetcher(dm)

Checks whether the data manager fetches a sequence instead of indexing. This is based on the source table. :returns True if dm is a fetcher. False if it is not.

get_dm_jobs(dm)

Gets the job entries for a single dm. Puts entries that already present in skipped_job_list. :returns job_list, skipped_job_list

initiate_job_lists()

Determines which data managers should be run to populate the data tables. Distinguishes between fetch jobs (download files) and index jobs. :return: populate self.fetch_jobs, self.skipped_fetch_jobs, self.index_jobs and self.skipped_index_jobs

input_entries_exist_in_data_tables(data_tables, input_dict)

Checks whether name and value entries from the input are already present in the data tables. If an entry is missing in of the tables, this function returns False

parse_items(items)

Parses items with jinja2. :param items: the items to be parsed :return: the parsed items

run(log=None, ignore_errors=False, overwrite=False, data_manager_mode: Literal['dry_run', 'populate', 'bundle'] = 'populate', history_name: str | None = None)

Runs the data managers. :param log: The log to be used. :param ignore_errors: Ignore erroring data_managers. Continue regardless. :param overwrite: Overwrite existing entries in data tables

ephemeris.run_data_managers.get_first_valid_entry(input_dict, key_list)

Iterates over key_list and returns the value of the first key that exists in the dictionary. Or returns None

ephemeris.run_data_managers.main(argv=None)

ephemeris.run_data_managers.wait(gi, job_list, log)

Waits until all jobs in a list are finished or failed. It will check the state of the created datasets every 30s. It will return a tuple: ( finished_jobs, failed_jobs )

ephemeris.set_library_permissions module

Tool to set permissions for all datasets of a given Galaxy Data Library

ephemeris.set_library_permissions.get_datasets(gi, library_id) → List[str]

ephemeris.set_library_permissions.main(argv=None)

ephemeris.set_library_permissions.set_permissions(gi, library_id, role_ids, auto)

ephemeris.setup_data_libraries module

Tool to setup data libraries on a galaxy instance

ephemeris.setup_data_libraries.create_batch_api(gi, desc)

ephemeris.setup_data_libraries.create_legacy(gi, desc)

ephemeris.setup_data_libraries.main(argv=None)

ephemeris.setup_data_libraries.setup_data_libraries(gi, data, training=False, legacy=False)

Load files into a Galaxy data library. By default all test-data tools from all installed tools will be linked into a data library.

ephemeris.shed_tools module

A tool to automate installation of tool repositories from a Galaxy Tool Shed into an instance of Galaxy.

Shed-tools has three commands: update, test and install.

Update simply updates all the tools in a Galaxy given connection details on the command line.

Test tests the specified tools in the Galaxy Instance.

Install allows installation of tools in multiple ways. Galaxy instance details and the installed tools can be provided in one of three ways:

1. In the YAML format via dedicated files (a sample can be found here).

2. On the command line as dedicated script options (see the usage help).

3. As a single composite parameter to the script. The parameter must be a single, YAML-formatted string with the keys corresponding to the keys available for use in the YAML formatted file (for example: –yaml_tool “{‘owner’: ‘kellrott’, ‘tool_shed_url’: ‘https://testtoolshed.g2.bx.psu.edu’, ‘tool_panel_section_id’: ‘peak_calling’, ‘name’: ‘synapse_interface’}”).

Only one of the methods can be used with each invocation of the script but if more than one are provided are provided, precedence will correspond to order of the items in the list above. When installing tools, Galaxy expects any tool_panel_section_id provided when installing a tool to already exist in the configuration. If the section does not exist, the tool will be installed outside any section. See shed_tool_conf.xml.sample in this directory for a sample of such file. Before running this script to install the tools, make sure to place such file into Galaxy’s configuration directory and set Galaxy configuration option tool_config_file to include it.

class ephemeris.shed_tools.FilterResults(already_installed_repos, not_installed_repos)

Bases: NamedTuple

already_installed_repos: List[InstallRepoDict]

Alias for field number 0

not_installed_repos: List[InstallRepoDict]

Alias for field number 1

class ephemeris.shed_tools.InstallRepoDict

Bases: TypedDict

changeset_revision: str | None

install_repository_dependencies: bool

install_resolver_dependencies: bool

install_tool_dependencies: bool

name: str

owner: str

revisions: List[str]

tool_panel_section_id: str | None

tool_panel_section_label: str | None

tool_shed_url: str

class ephemeris.shed_tools.InstallRepositoryManager(galaxy_instance)

Bases: object

Manages the installation of new repositories on a galaxy instance

filter_installed_repos(repos: Iterable[InstallRepoDict], check_revision: bool = True) → FilterResults

This filters a list of repositories

install_repositories(repositories: ~typing.List[~ephemeris.shed_tools.InstallRepoDict], log=<Logger ephemeris.shed_tools (WARNING)>, force_latest_revision: bool = False, default_toolshed: str = 'https://toolshed.g2.bx.psu.edu/', default_install_tool_dependencies: bool = False, default_install_resolver_dependencies: bool = True, default_install_repository_dependencies: bool = True)

Install a list of tools on the current galaxy

install_repository_revision(repository: InstallRepoDict, log)

installed_repositories() → List[InstallRepoDict]

Get currently installed tools

test_tools(test_json, repositories=None, log=<Logger ephemeris.shed_tools (WARNING)>, test_user_api_key=None, test_user='ephemeris@galaxyproject.org', test_history_name=None, parallel_tests=1, test_all_versions=False, client_test_config_path=None)

Run tool tests for all tools in each repository in supplied tool list or self.installed_repositories().

update_repositories(repositories=None, log=<Logger ephemeris.shed_tools (WARNING)>, **kwargs)

wait_for_install(repository, log=<Logger ephemeris.shed_tools (WARNING)>, timeout=3600)

If nginx times out, we look into the list of installed repositories and try to determine if a repository of the same namer/owner is still installing. Returns True if install finished successfully, returns False when timeout is exceeded or installation has failed.

class ephemeris.shed_tools.InstallResults(installed_repositories, skipped_repositories, errored_repositories)

Bases: NamedTuple

errored_repositories: List[InstallRepoDict]

Alias for field number 2

installed_repositories: List[InstallRepoDict]

Alias for field number 0

skipped_repositories: List[InstallRepoDict]

Alias for field number 1

ephemeris.shed_tools.args_to_repos(args) → List[InstallRepoDict]

ephemeris.shed_tools.log_repository_install_error(repository, start, msg, log)

Log failed repository installations. Return a dictionary with information

ephemeris.shed_tools.log_repository_install_skip(repository, counter, total_num_repositories, log)

ephemeris.shed_tools.log_repository_install_start(repository: InstallRepoDict, counter, total_num_repositories, installation_start, log)

ephemeris.shed_tools.log_repository_install_success(repository, start, log)

Log successful repository installation. Repositories that finish in error still count as successful installs currently.

ephemeris.shed_tools.main(argv=None)

ephemeris.shed_tools_args module

This file contains the parser for shed_tools

ephemeris.shed_tools_args.parser()

construct the parser object

ephemeris.shed_tools_methods module

ephemeris.shed_tools_methods.complete_repo_information(tool: InstallRepoDict, default_toolshed_url: str, default_install_tool_dependencies: bool, default_install_repository_dependencies: bool, default_install_resolver_dependencies: bool, force_latest_revision) → InstallRepoDict

ephemeris.shed_tools_methods.flatten_repo_info(repositories: Iterable[InstallRepoDict]) → List[InstallRepoDict]

Flatten the dict containing info about what tools to install. The tool definition YAML file allows multiple revisions to be listed for the same tool. To enable simple, iterative processing of the info in this script, flatten the tools_info list to include one entry per tool revision.

Parameters:

repositories (list of dicts) – Each dict in this list should contain info about a tool.

Return type:

list of dicts

Returns:

Return a list of dicts that correspond to the input argument such that if an input element contained revisions key with multiple values, those will be returned as separate list items.

ephemeris.shed_tools_methods.format_tool_shed_url(tool_shed_url: str) → str

ephemeris.shed_tools_methods.get_changeset_revisions(repository: InstallRepoDict, force_latest_revision: bool = False)

Select the correct changeset revision for a repository, and make sure the repository exists (i.e a request to the tool shed with name and owner returns a list of revisions). Return repository or None, if the repository could not be found on the specified tool shed.

ephemeris.sleep module

Utility to do a blocking sleep until a Galaxy instance is responsive. This is useful in docker images, in RUN steps, where one needs to wait for a currently starting Galaxy to be alive, before API requests can be made successfully. The script functions by making repeated requests to http(s)://fqdn/api/version, an API which requires no authentication to access.

class ephemeris.sleep.SleepCondition

Bases: object

cancel()

ephemeris.sleep.galaxy_wait(galaxy_url, verbose=False, timeout=0, sleep_condition=None, api_key=None, ensure_admin=False)

Pass user_key to ensure it works before returning.

ephemeris.sleep.main(argv=None)

Main function

ephemeris.workflow_install module

Tool to install workflows on a Galaxy instance.

ephemeris.workflow_install.import_workflow(gi, path, publish_wf=False)

Given a connection to a Galaxy Instance (gi) and a path to a Galaxy workflow file, this function will import the worklfow into Galaxy.

ephemeris.workflow_install.main(argv=None)

This script uses bioblend to import .ga workflow files into a running instance of Galaxy

Module contents

ephemeris.check_url(url, log=None)

ephemeris.dump_to_yaml_file(content, file_name)

Dump YAML-compatible content to file_name.

ephemeris.get_galaxy_connection(args, file=None, log=None, login_required=True)

Return a Galaxy connection, given a user or an API key. If not given gets the arguments from the file. If either is missing raise ValueError.

ephemeris.get_or_create_history(history_name: str, gi: GalaxyInstance)

ephemeris.load_yaml_file(filename)

Load YAML from the tool_list_file and return a dict with the content.