ETL Services in APC Gateway

The Extract, Transform and Load (ETL) services, which reside in the NOVUS Service Shell, are at the core of APC Gateway.

The following is a summary of the ETL services and their functions within APC Gateway:

  1. Read data: Read data through an XML parser.
  2. Validate data:
    • Stage one: Capture obvious data exceptions

    • Stage two: Throw out data subset based on missing data elements or replace missing data elements

  3. Persist data exceptions and errors in hold data storage: Data storage is logically separate from PLAN database for review by Trapeze user.

  4. Clean data: The data cleaning algorithm attempts to accurately balance boarding and alighting data to clean and shape ridership data.

  5. Transform data: While mapping data from APC and fare systems to Trapeze applications, the data format may be corrected (for example, integer to decimal).

  6. Persist accepted data in Trapeze PLAN database.

  7. Review data: You can navigate data exceptions/errors, compare with correct data, and choose to modify, accept or reject data exceptions within APC Gateway.