nexusLIMS.harvesters.nemo package¶

NEMO harvester module.

This module contains the functionality to harvest instruments, reservations, etc. from an instance of NEMO (https://github.com/usnistgov/NEMO/), a calendering and laboratory logistics application.

nexusLIMS.harvesters.nemo.res_event_from_session(session: Session) → ReservationEvent[source]¶

Create reservation event from session.

Create an internal ReservationEvent representation of a session by finding a matching reservation in the NEMO system and parsing the data contained within into a ReservationEvent.

This method assumes a certain format for the “reservation questions” associated with each reservation and parses that information into the resulting ReservationEvent. The most critical of these is the data_consent field. If an affirmative response in this field is not found (because the user declined consent or the reservation questions are missing), a record will not be built.

The following JSON object represents a minimal schema for a set of NEMO “Reservation Questions” that will satisfy the expectations of this method. Please see the NEMO documentation on this feature for more details.

[
  {
    "type": "textbox",
    "name": "project_id",
    "title": "Project ID",
  },
  {
    "type": "textbox",
    "name": "experiment_title",
    "title": "Title of Experiment",
  },
  {
    "type": "textarea",
    "name": "experiment_purpose",
    "title": "Experiment Purpose",
  },
  {
    "type": "radio",
    "title": "Agree to NexusLIMS curation",
    "choices": ["Agree", "Disagree"],
    "name": "data_consent",
    "default_choice": "Agree"
  },
  {
    "type": "group",
    "title": "Sample information",
    "name": "sample_group",
    "questions": [
      {
        "type": "textbox",
        "name": "sample_name",
        "title": "Sample Name / PID",
      },
      {
        "type": "radio",
        "title": "Sample or PID?",
        "choices": ["Sample Name", "PID"],
        "name": "sample_or_pid",
      },
      {
        "type": "textarea",
        "name": "sample_details",
        "title": "Sample Details",
      }
    ]
  }
]

Parameters: session – The session for which to get a reservation event
Returns: res_event – The matching reservation event
Return type: ReservationEvent

Submodules¶

nexusLIMS.harvesters.nemo.connector module¶

Defines the NemoConnector class that is used to interface with the NEMO API.

class nexusLIMS.harvesters.nemo.connector.NemoConnector(base_url: str, token: str, strftime_fmt: Optional[str] = None, strptime_fmt: Optional[str] = None, timezone: Optional[str] = None)[source]¶

Bases: object

A connection to an instance of the API of the NEMO laboratory management software.

Provides helper methods for fetching data from the API.

Parameters

base_url (str) – The “root” of the API including a trailing slash; e.g. ‘https://nemo.url.com/api/’
token (str) – An authentication token for this NEMO instance
strftime_fmt (str) – The “date format” to use when encoding dates to send as filters to the NEMO API. Should follow the same convention as strftime() and strptime() Behavior. If None, ISO 8601 format will be used.
strptime_fmt (str) – The “date format” to use when decoding date strings received in the response from the API. Should follow the same convention as strftime() and strptime() Behavior. If None, ISO 8601 format will be used.
timezone (str) – The timezone to use when decoding date strings received in the response from the API. Should be a IANA time zone database string; e.g. “America/New_York”. Useful if no timezone information is returned from an instance of the NEMO API. If None, no timezone setting will be done and the code will use whatever was returned from the server as is.

get_known_tool_ids() → List[int][source]¶

Get NEMO tool ID values from the NexusLIMS database.

Inspect the api_url values of known Instruments (from the instruments table in the DB), and extract their tool_id number if it is from this NemoConnector.

Returns: tool_ids – The list of tool ID numbers known to NexusLIMS for this harvester
Return type: List[int]

get_projects(proj_id: Union[int, List[int]]) → List[Dict][source]¶

Get a list of one or more projects from the NEMO API in a dictionary.

The local cache will be checked prior to fetching from the API to save a network request if possible.

Parameters: proj_id – The project(s) to fetch, as indexed by the NEMO instance (i.e. proj_id should be the internal primary key used by NEMO to identify the project. If an empty list is given, all projects will be returned.
Returns: projects – A list (could be empty) of projects that match the id (or ids) given in proj_id
Return type: List[Dict]
Raises: HTTPError – Raised if the request to the API did not go through correctly

get_reservations(dt_from: Optional[datetime] = None, dt_to: Optional[datetime] = None, tool_id: Optional[Union[int, List[int]]] = None, *, cancelled: Optional[bool] = False) → List[Dict][source]¶

Get reservations from the NEMO API filtered in various ways.

Return a list of reservations from the API, filtered by date (inclusive). If only one argument is provided, the API will return all reservations either before or after the parameter. With no arguments, the method will return all reservations. The method will “auto-expand” linked information such as user, project, tool, etc. so results will have a full dictionary for each of those fields, rather than just the index (as returned from the API).

Parameters

dt_from – The “starting point” of the time range; only reservations at or after this point in time will be returned
dt_to – The “ending point” of the time range; only reservations at or prior to this point in time will be returned
tool_id – A tool identifier (or list of them) to limit the scope of the reservation search (this should be the NEMO internal integer ID)
cancelled – Whether to get canceled or active reservations in the response (default is False – meaning non-cancelled active reservations are returned by default). Set to None to include all reservations regardless of whether they are cancelled or active.

Returns

reservations – A list (could be empty) of reservations that match the date range supplied

Return type

List[Dict]

get_session_from_usage_event(event_id: int) → Optional[Session][source]¶

Get a Session representation of a usage event.

Get a Session representation of a usage event for use in dry runs of the record builder.

Parameters: event_id – The NEMO id number for the event to insert
Returns: session – A representation of the usage_event from NEMO as a Session object
Return type: Session

get_tools(tool_id: Union[int, List[int]]) → List[Dict][source]¶

Get a list of one or more tools from the NEMO API in a dictionary.

Parameters: tool_id – The tool(s) to fetch, as indexed by the NEMO instance (i.e. tool_id should be the internal primary key used by NEMO to identify the tool. If an empty list is given, all tools will be returned.
Returns: tools – A list (could be empty) of tools that match the id (or ids) given in tool_id
Return type: List[Dict]
Raises: HTTPError – Raised if the request to the API did not go through correctly

get_usage_events(event_id: Optional[Union[int, List[int]]] = None, user: Optional[Union[str, int]] = None, dt_range: Optional[Tuple[Optional[datetime], Optional[datetime]]] = None, tool_id: Optional[Union[int, List[int]]] = None) → List[source]¶

Get usage events from the NEMO API filtered in various ways.

Return a list of usage events from the API, filtered by date (inclusive). If only one argument is provided, the API will return all reservations either before or after the parameter. With no arguments, the method will return all reservations. The method will “auto-expand” linked information such as user, project, tool, etc. so results will have a full dictionary for each of those fields, rather than just the index (as returned from the API).

Parameters

event_id – The NEMO integer identifier (or a list of them) to fetch. If None, the returned usage events will not be filtered by ID number
user – The user for which to fetch usage events, as either an integer representing their id in the NEMO instance, or as a string containing their username. If None is given, usage events from all users will be returned.
dt_range – The “starting” and “end” points of the time range; only usage events starting between these points in time will be returned
tool_id – A tool identifier (or list of them) to limit the scope of the usage event search (this should be the NEMO internal integer ID). Regardless of what value is given, this method will always limit the API query to tools specified in the NexusLIMS DB for this harvester

Returns

usage_events – A list (could be empty) of usage events that match the filters supplied

Return type

List

get_users(user_id: Optional[Union[int, List[int]]] = None) → List[Dict][source]¶

Get a list of one or more users from the NEMO API in a dictionary.

The results will be cached in the NemoConnector instance to prevent multiple API queries if not necessary.

Parameters: user_id – The user(s) to fetch, as indexed by the NEMO instance (i.e. user_id should be the internal primary key used by NEMO to identify the user. If an empty list or None is given, all users will be returned.
Returns: users – A list (could be empty) of users that match the ids and/or usernames given
Return type: List[Dict]
Raises: HTTPError – Raised if the request to the API did not go through correctly

get_users_by_username(username=None) → List[source]¶

Get a list of one or more users from the NEMO API in a dictionary.

The results will be cached in the NemoConnector instance to prevent multiple API queries if not necessary.

Parameters: username (str or list of str) – The user(s) to fetch, as indexed by their usernames in the NEMO instance. If an empty list or None is given, all users will be returned.
Returns: users – A list (could be empty) of users that match the ids and/or usernames given
Return type: list
Raises: HTTPError – Raised if the request to the API did not go through correctly

projects: Dict[int, Dict]¶

strftime(date_dt) → str[source]¶

Convert datetime to appropriate string format for this connector.

Using the settings for this NemoConnector, convert a datetime object to a string that will be understood by the API. If the strftime_fmt attribute for this NemoConnector is None, ISO 8601 format will be used.

Parameters: date_dt – The date to be converted as a datetime object
Returns: date_str – The date formatted as a string that will be understandable by the API for this NemoConnector
Return type: str

strptime(date_str) → datetime[source]¶

Convert string to datetime using this connector’s API date format.

Using the settings for this NemoConnector, convert a datetime string representation from the API into a datetime object that can be used in Python. If the strptime_fmt attribute for this NemoConnector is None, ISO 8601 format will be assumed. If a timezone is specified for this server, the resulting datetime will be coerced to that timezone.

Parameters: date_str – The date formatted as a string that is returned by the API for this NemoConnector
Returns: date_dt – The date to be converted as a datetime object
Return type: datetime

tools: Dict[int, Dict]¶

users: Dict[int, Dict]¶

users_by_username: Dict[str, Dict]¶

write_usage_event_to_session_log(event_id: int) → None[source]¶

Write a usage event to the NexusLIMS database session log.

Inserts two rows (if needed) into the session_log (marking the start and end of a usage event), only for instruments recognized by NexusLIMS (i.e. that have a row in the instruments table of the DB). If the usage event has not ended yet, no action is performed.

Parameters: event_id – The NEMO id number for the event to insert

nexusLIMS.harvesters.nemo.exceptions module¶

Contains some exceptions specific to NEMO harvesting.

exception nexusLIMS.harvesters.nemo.exceptions.NoDataConsentError[source]¶

Bases: Exception

A user has not given their consent to have data harvested.

exception nexusLIMS.harvesters.nemo.exceptions.NoMatchingReservationError[source]¶

Bases: Exception

No matching reservation (we cannot assume consent to harvest data).

nexusLIMS.harvesters.nemo.utils module¶

Various utility functions used by the NEMO harvester.

nexusLIMS.harvesters.nemo.utils.add_all_usage_events_to_db(user: Optional[Union[str, int]] = None, dt_from: Optional[datetime] = None, dt_to: Optional[datetime] = None, tool_id: Optional[Union[int, List[int]]] = None)[source]¶

Add all usage events to database for enabled NEMO connectors.

Loop through enabled NEMO connectors and add each one’s usage events to the NexusLIMS session_log database table (if required).

Parameters

user – The user(s) for which to add usage events. If None, events will not be filtered by user at all
dt_from – The point in time after which usage events will be added. If None, no date filtering will be performed
dt_to – The point in time before which usage events will be added. If None, no date filtering will be performed
tool_id – The tools(s) for which to add usage events. If 'None' (default), the tool IDs for each instrument in the NexusLIMS DB will be extracted and used to limit the API response

nexusLIMS.harvesters.nemo.utils.get_connector_by_base_url(base_url: str) → NemoConnector[source]¶

Get an enabled NemoConnector by inspecting the base_url.

Parameters: base_url – A portion of the API url to search for
Returns: n – The enabled NemoConnector instance
Return type: NemoConnector
Raises: LookupError – Raised if a matching connector is not found

nexusLIMS.harvesters.nemo.utils.get_connector_for_session(session: Session) → NemoConnector[source]¶

Get the appropriate NEMO connector for a given Session.

Given a Session, find the matching NemoConnector from the enabled list of NEMO harvesters.

Parameters: session – The session for which a NemoConnector is needed
Returns: n – The connector object that allows for querying the NEMO API for the instrument contained in session
Return type: NemoConnector
Raises: LookupError – Raised if a matching connector is not found

nexusLIMS.harvesters.nemo.utils.get_harvesters_enabled() → List[NemoConnector][source]¶

Return a list of enabled connectors based off the environment.

Returns: harvesters_enabled – A list of NemoConnector objects representing the NEMO APIs enabled via environment settings
Return type: List[NemoConnector]

nexusLIMS.harvesters.nemo.utils.get_usage_events_as_sessions(user: Optional[Union[str, int]] = None, dt_from: Optional[datetime] = None, dt_to: Optional[datetime] = None, tool_id: Optional[Union[int, List[int]]] = None) → List[Session][source]¶

Get all usage events for enabled NEMO connectors as Sessions.

Loop through enabled NEMO connectors and return each one’s usage events to as Session objects without writing logs to the session_log table. Mostly used for doing dry runs of the record builder.

Parameters

user – The user(s) for which to fetch usage events. If None, events will not be filtered by user at all
dt_from – The point in time after which usage events will be fetched. If None, no date filtering will be performed
dt_to – The point in time before which usage events will be fetched. If None, no date filtering will be performed
tool_id – The tools(s) for which to fetch usage events. If None, events will only be filtered by tools known in the NexusLIMS DB for each connector

nexusLIMS.harvesters.nemo.utils.id_from_url(url: str) → Optional[int][source]¶

Get the value of the id query parameter stored in URL string.

This is used to extract the value as needed from API strings.

Parameters: url – The URL to parse, such as https://nemo.url.com/api/usage_events/?id=9
Returns: this_id – The id value if one is present, otherwise None
Return type: None or int

nexusLIMS.harvesters.nemo.utils.process_res_question_samples(res_dict: Dict) → Tuple[Optional[List[Optional[str]]], Optional[List[Optional[str]]], Optional[List[Optional[str]]], Optional[List[Optional[str]]]][source]¶

Process sample information from reservation questions.

Parameters: res_dict – The reservation dictionary (i.e. the response from the reservations api endpoint)

NexusLIMS Documentation

Page content

nexusLIMS.harvesters.nemo package¶

Submodules¶

nexusLIMS.harvesters.nemo.connector module¶

nexusLIMS.harvesters.nemo.exceptions module¶

nexusLIMS.harvesters.nemo.utils module¶