Introduction

To evaluate business architecture data effectively, business users follow a structured rating process for each Business Architecture domain within their document. The process involves assessing three key aspects: relevance, quality, and quantity.

How it works

1. Relevance: Users determine how closely the domain data aligns with the Organization's strategic goals. Ratings range from "Very High" (direct alignment) to "Very Low" (minimal support), with an option for "Incomplete" if no relevant data exists.

2. Quality: This assesses the documentation's adherence to industry standards, specifically ISO 9001 for documents. Ratings span from "Very High" (meets/exceeds Level 5 standards) to "Very Low" (meets Level 1 standards), with "Incomplete" indicating non-compliance with any standard. For details, see: https://orthogramic.atlassian.net/wiki/spaces/OUG/pages/edit-v2/233078785?draftShareId=13f5f7a2-d3bb-4d7d-bee5-6647f2aeed12#Assessing-document-quality-in-terms-of-ISO-9001

3. Quantity: Users evaluate the comprehensiveness of the documentation against ISO 9001 for documents requirements. The scale goes from "Very High" (surpassing Level 5 requirements) to "Very Low" (meeting Level 1 requirements), with "Incomplete" for insufficient documentation.

By systematically rating each domain, users ensure a thorough and objective evaluation of how well their business architecture data supports strategic goals and meets documentation standards. This structured approach helps in identifying strengths and areas for improvement, ultimately aiding in more informed decision-making.

Example document criteria analysis

This is an example of advanced criteria analysis during document editing.

Assessing document quality in terms of ISO 9001

Review Structure and Content

1. Quality Manual: Does the document include a quality manual that outlines the scope of the QMS, including any exclusions?

2. Quality Policy and Objectives: Are the Organization's quality policy and objectives clearly stated?

3. Procedures and Work Instructions: Are there documented procedures and detailed work instructions that ensure process consistency?

Control of Documented Information

1. Approval: Check if the document has been approved for adequacy prior to use.

   • Review and Update: Look for evidence of regular reviews and updates, including re-approval if necessary.

   • Identification and Distribution: Ensure that the document is clearly identified and distributed to relevant personnel.

Accessibility and Storage

1. Accessibility: Confirm that relevant personnel have easy access to the document.

   • Storage and Protection: Verify that the document is stored and protected to prevent damage or loss.

Retention and Disposal

1. Retention: Check that documents are retained for the required period.

   • Disposal: Ensure there are procedures for the disposal of documents when no longer needed.

Accuracy and Currency

1. Accuracy: Assess if the document is accurate and reflects the current state of the QMS.

   • Currency: Check if the document is up-to-date and includes any recent changes or updates.

Compliance

1. Regulatory Requirements: Ensure the document complies with relevant regulatory and statutory requirements.

Quality assessment criteria

Normalisation

Normalisation is a critical step in Document Criteria Analysis to ensure consistency and fairness when evaluating documents with different characteristics. It standardises raw scores across multiple criteria, making them comparable within a unified scale. This approach aligns with the methodology used in Document Weighting to provide balanced and reliable document assessments.

Purpose of Normalisation

• Standardises Scores: Adjusts document criteria scores to a common scale.

• Ensures Fair Comparisons: Allows documents of varying lengths, complexity, and relevance to be assessed equitably.

• Enhances Consistency: Aligns document evaluations with the weighting model used in Document Weighting.

Normalisation Methodology

The normalised score for each document criterion is computed using the Min-Max normalisation formula:

Where:

• is the raw score for a criterion.

• and are the minimum and maximum scores across all documents for that criterion.

• results in a value between 0 and 1, ensuring comparability.

Alternatively, Z-score normalisation may be used for datasets where distributions vary significantly:

Where:

• is the mean score.

• is the standard deviation.

• This method ensures scores follow a standard normal distribution with a mean of 0 and a standard deviation of 1.

Integration with Document Weighting

• The normalised criteria scores serve as inputs for Document Weighting, ensuring the weighted score computation remains consistent across document evaluations.

• By normalising scores before weighting, the document evaluation process maintains robustness, preventing bias due to variations in document characteristics.

Example

After normalisation, these values are used in the final document weighting calculations, ensuring consistent assessment criteria.

Conclusion

By incorporating normalisation into Document Criteria Analysis, Orthogramic ensures a structured and unbiased document evaluation process.