ephemeris package¶
Submodules¶
ephemeris.common_parser module¶
- class ephemeris.common_parser.ArgumentDefaultsHideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]¶
Bases:
HideUnderscoresHelpFormatter
,ArgumentDefaultsHelpFormatter
- class ephemeris.common_parser.HideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]¶
Bases:
HelpFormatter
- class ephemeris.common_parser.RawDescriptionHideUnderscoresHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]¶
Bases:
HideUnderscoresHelpFormatter
,RawDescriptionHelpFormatter
ephemeris.ephemeris_log module¶
- class ephemeris.ephemeris_log.ProgressConsoleHandler(stream=None)[source]¶
Bases:
StreamHandler
A handler class which allows the cursor to stay on one line for selected messages
- emit(record)[source]¶
Emit a record.
If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.
- on_same_line = False¶
ephemeris.generate_tool_list_from_ga_workflow_files module¶
Tool to generate tools from workflows
- ephemeris.generate_tool_list_from_ga_workflow_files.extract_tool_shed_repositories_from_workflow_dict(workflow_dictionary)[source]¶
- ephemeris.generate_tool_list_from_ga_workflow_files.generate_repo_list_from_workflow(workflow_files: Iterable[str], panel_label: str) List[InstallRepoDict] [source]¶
- ephemeris.generate_tool_list_from_ga_workflow_files.generate_tool_list_from_workflow(workflow_files: Iterable[str], panel_label: str, output_file: str)[source]¶
- Return type:
object
- ephemeris.generate_tool_list_from_ga_workflow_files.print_yaml_tool_list(tool_dictionary, output_file)[source]¶
- ephemeris.generate_tool_list_from_ga_workflow_files.reduce_tool_list(tool_list: List[InstallRepoDict]) List[InstallRepoDict] [source]¶
- ephemeris.generate_tool_list_from_ga_workflow_files.translate_workflow_dictionary_to_tool_list(workflow_dictionary, panel_label: str) List[InstallRepoDict] [source]¶
ephemeris.get_tool_list_from_galaxy module¶
Tool to extract a tool list from galaxy.
- class ephemeris.get_tool_list_from_galaxy.GiToToolYaml(gi, include_tool_panel_section_id=False, skip_tool_panel_section_name=True, skip_changeset_revision=False, get_data_managers=False, get_all_tools=False)[source]¶
Bases:
object
- property installed_tool_list¶
gets a tool list from the toolclient :return:
- property repository_list¶
Toolbox elements returned by api/tools may be of class ToolSection or Tool. Parse these accordingly to get a list of repositories.
- property tool_list¶
- property toolbox¶
Gets the toolbox elements from <galaxy_url>/api/tools
- ephemeris.get_tool_list_from_galaxy.get_repo_from_tool(tool)[source]¶
Get the minimum items required for re-installing a (list of) tools
- ephemeris.get_tool_list_from_galaxy.merge_repository_changeset_revisions(repository_list)[source]¶
Each installed changeset revision of a tool is listed individually. Merge revisions of the same tool into a list.
- ephemeris.get_tool_list_from_galaxy.the_same_repository(repo_1_info, repo_2_info, check_revision=True)[source]¶
Given two dicts containing info about repositories, determine if they are the same repository. Each of the dicts must have the following keys: changeset_revisions`( if check revisions is true), `name, owner, and (either tool_shed or tool_shed_url).
ephemeris.install_tool_deps module¶
Tool to install tool dependencies on a Galaxy instance.
ephemeris.run_data_managers module¶
Run-data-managers is a tool for provisioning data on a galaxy instance.
Run-data-managers has the ability to run multiple data managers that are interdependent. When a reference genome is needed for bwa-mem for example, Run-data-managers can first run a data manager to fetch the fasta file and run another data manager that indexes the fasta file for bwa-mem. This functionality depends on the “watch_tool_data_dir” setting in galaxy.ini to be True. Also, if a new data manager is installed, galaxy needs to be restarted in order for it’s tool_data_dir to be watched.
Run-data-managers needs a yaml that specifies what data managers are run and with which settings. Example files can be found here, here, and here.
By default run-data-managers skips entries in the yaml file that have already been run. It checks it in the following way:
If the data manager has input variables “name” or “sequence_name” it will check if the “name” column in the data table already has this entry. “name” will take precedence over “sequence_name”.
If the data manager has input variables “value”, “sequence_id” or ‘dbkey’ it will check if the “value” column in the data table already has this entry. Value takes precedence over sequence_id which takes precedence over dbkey.
If none of the above input variables are specified the data manager will always run.
- class ephemeris.run_data_managers.DataManagers(galaxy_instance: GalaxyInstance, configuration)[source]¶
Bases:
object
- data_table_entry_exists(data_table_name, entry, column='value')[source]¶
Checks whether an entry exists in the a specified column in the data_table.
- dm_is_fetcher(dm)[source]¶
Checks whether the data manager fetches a sequence instead of indexing. This is based on the source table. :returns True if dm is a fetcher. False if it is not.
- get_dm_jobs(dm)[source]¶
Gets the job entries for a single dm. Puts entries that already present in skipped_job_list. :returns job_list, skipped_job_list
- initiate_job_lists()[source]¶
Determines which data managers should be run to populate the data tables. Distinguishes between fetch jobs (download files) and index jobs. :return: populate self.fetch_jobs, self.skipped_fetch_jobs, self.index_jobs and self.skipped_index_jobs
- input_entries_exist_in_data_tables(data_tables, input_dict)[source]¶
Checks whether name and value entries from the input are already present in the data tables. If an entry is missing in of the tables, this function returns False
- parse_items(items)[source]¶
Parses items with jinja2. :param items: the items to be parsed :return: the parsed items
- run(log=None, ignore_errors=False, overwrite=False, data_manager_mode: Literal['dry_run', 'populate', 'bundle'] = 'populate', history_name: str | None = None)[source]¶
Runs the data managers. :param log: The log to be used. :param ignore_errors: Ignore erroring data_managers. Continue regardless. :param overwrite: Overwrite existing entries in data tables
ephemeris.set_library_permissions module¶
Tool to set permissions for all datasets of a given Galaxy Data Library
ephemeris.setup_data_libraries module¶
Tool to setup data libraries on a galaxy instance
ephemeris.shed_tools module¶
A tool to automate installation of tool repositories from a Galaxy Tool Shed into an instance of Galaxy.
Shed-tools has three commands: update, test and install.
Update simply updates all the tools in a Galaxy given connection details on the command line.
Test tests the specified tools in the Galaxy Instance.
Install allows installation of tools in multiple ways. Galaxy instance details and the installed tools can be provided in one of three ways:
In the YAML format via dedicated files (a sample can be found here).
On the command line as dedicated script options (see the usage help).
As a single composite parameter to the script. The parameter must be a single, YAML-formatted string with the keys corresponding to the keys available for use in the YAML formatted file (for example: –yaml_tool “{‘owner’: ‘kellrott’, ‘tool_shed_url’: ‘https://testtoolshed.g2.bx.psu.edu’, ‘tool_panel_section_id’: ‘peak_calling’, ‘name’: ‘synapse_interface’}”).
Only one of the methods can be used with each invocation of the script but if more than one are provided are provided, precedence will correspond to order of the items in the list above. When installing tools, Galaxy expects any tool_panel_section_id provided when installing a tool to already exist in the configuration. If the section does not exist, the tool will be installed outside any section. See shed_tool_conf.xml.sample in this directory for a sample of such file. Before running this script to install the tools, make sure to place such file into Galaxy’s configuration directory and set Galaxy configuration option tool_config_file to include it.
- class ephemeris.shed_tools.FilterResults(already_installed_repos, not_installed_repos)[source]¶
Bases:
NamedTuple
- already_installed_repos: List[InstallRepoDict]¶
Alias for field number 0
- not_installed_repos: List[InstallRepoDict]¶
Alias for field number 1
- class ephemeris.shed_tools.InstallRepoDict[source]¶
Bases:
TypedDict
- changeset_revision: str | None¶
- install_repository_dependencies: bool¶
- install_resolver_dependencies: bool¶
- install_tool_dependencies: bool¶
- name: str¶
- owner: str¶
- revisions: List[str]¶
- tool_panel_section_id: str | None¶
- tool_panel_section_label: str | None¶
- tool_shed_url: str¶
- class ephemeris.shed_tools.InstallRepositoryManager(galaxy_instance)[source]¶
Bases:
object
Manages the installation of new repositories on a galaxy instance
- filter_installed_repos(repos: Iterable[InstallRepoDict], check_revision: bool = True) FilterResults [source]¶
This filters a list of repositories
- install_repositories(repositories: ~typing.List[~ephemeris.shed_tools.InstallRepoDict], log=<Logger ephemeris.shed_tools (WARNING)>, force_latest_revision: bool = False, default_toolshed: str = 'https://toolshed.g2.bx.psu.edu/', default_install_tool_dependencies: bool = False, default_install_resolver_dependencies: bool = True, default_install_repository_dependencies: bool = True)[source]¶
Install a list of tools on the current galaxy
- install_repository_revision(repository: InstallRepoDict, log)[source]¶
- installed_repositories() List[InstallRepoDict] [source]¶
Get currently installed tools
- test_tools(test_json, repositories=None, log=<Logger ephemeris.shed_tools (WARNING)>, test_user_api_key=None, test_user='ephemeris@galaxyproject.org', test_history_name=None, parallel_tests=1, test_all_versions=False, client_test_config_path=None)[source]¶
Run tool tests for all tools in each repository in supplied tool list or
self.installed_repositories()
.
- update_repositories(repositories=None, log=<Logger ephemeris.shed_tools (WARNING)>, **kwargs)[source]¶
- wait_for_install(repository, log=<Logger ephemeris.shed_tools (WARNING)>, timeout=3600)[source]¶
If nginx times out, we look into the list of installed repositories and try to determine if a repository of the same namer/owner is still installing. Returns True if install finished successfully, returns False when timeout is exceeded or installation has failed.
- class ephemeris.shed_tools.InstallResults(installed_repositories, skipped_repositories, errored_repositories)[source]¶
Bases:
NamedTuple
- errored_repositories: List[InstallRepoDict]¶
Alias for field number 2
- installed_repositories: List[InstallRepoDict]¶
Alias for field number 0
- skipped_repositories: List[InstallRepoDict]¶
Alias for field number 1
- ephemeris.shed_tools.args_to_repos(args) List[InstallRepoDict] [source]¶
- ephemeris.shed_tools.log_repository_install_error(repository, start, msg, log)[source]¶
Log failed repository installations. Return a dictionary with information
- ephemeris.shed_tools.log_repository_install_skip(repository, counter, total_num_repositories, log)[source]¶
- ephemeris.shed_tools.log_repository_install_start(repository: InstallRepoDict, counter, total_num_repositories, installation_start, log)[source]¶
ephemeris.shed_tools_args module¶
This file contains the parser for shed_tools
ephemeris.shed_tools_methods module¶
- ephemeris.shed_tools_methods.complete_repo_information(tool: InstallRepoDict, default_toolshed_url: str, default_install_tool_dependencies: bool, default_install_repository_dependencies: bool, default_install_resolver_dependencies: bool, force_latest_revision) InstallRepoDict [source]¶
- ephemeris.shed_tools_methods.flatten_repo_info(repositories: Iterable[InstallRepoDict]) List[InstallRepoDict] [source]¶
Flatten the dict containing info about what tools to install. The tool definition YAML file allows multiple revisions to be listed for the same tool. To enable simple, iterative processing of the info in this script, flatten the tools_info list to include one entry per tool revision.
- Parameters:
repositories (list of dicts) – Each dict in this list should contain info about a tool.
- Return type:
list of dicts
- Returns:
Return a list of dicts that correspond to the input argument such that if an input element contained revisions key with multiple values, those will be returned as separate list items.
- ephemeris.shed_tools_methods.get_changeset_revisions(repository: InstallRepoDict, force_latest_revision: bool = False)[source]¶
Select the correct changeset revision for a repository, and make sure the repository exists (i.e a request to the tool shed with name and owner returns a list of revisions). Return repository or None, if the repository could not be found on the specified tool shed.
ephemeris.sleep module¶
Utility to do a blocking sleep until a Galaxy instance is responsive.
This is useful in docker images, in RUN steps, where one needs to wait
for a currently starting Galaxy to be alive, before API requests can be
made successfully.
The script functions by making repeated requests to
http(s)://fqdn/api/version
, an API which requires no authentication
to access.
ephemeris.workflow_install module¶
Tool to install workflows on a Galaxy instance.