Data collection and analysis methods generate the evaluation outcome and recommendations from the full range of personnel:
Information system management and technical staff,
Users working in a finance and operations capacity,
Users working in a medical, clinical and nursing capacity, and
Users working in a research capacity.
Data Collection
Quantitative Method
Built-in questionnaires pose predefined questions that provide detailed data on the system's characteristics, use, and impact, with:
- Likert scales,
- Two-item scale,
- Open-ended questions, and
- Data entry of quantitative measures.
Qualitative Methods
Some factors must be assessed using qualitative methods:
- Document review: technical, financial and administrative documents, system and training documentation,
- System review: system testing, performance and usage monitoring, helpdesk logs, and
- Focus groups and interviews: built-in interview guides.
Data Analysis
Quantitative Data Analysis
The toolset safeguards and stores all content from the completed surveys. Statistical measures are produced incorporating the specific variables:
- Nominal variables: number and percentage of personnel per category, mean, mode; and
- Ordinal variables: minimum, maximum, range, median and quartiles.
Multiple answers and filtered questions are handled.
Weighing mechanisms are applied to compensate for over- and under-representation.
Statistical tests and procedures are performed on individual and groups of variables.
Graphical outputs are produced.
Qualitative Data Analysis
Interviews and focus groups are transcribed and analyzed by:
- Building matrices with categorized data;
- Producing displays such as flowcharts and graphs;
- Tabulating frequencies and exploring relationships;
- Situating information within a historical perspective.
The data from both documentation and system reviews is merged and the overall analysis explains the system’s use and impact.