Annotating Text from News Articles to Enhance the Performance of an AI Model

HabileData classified and labeled over 10,000 construction-related articles, ensuring rigorous validation, verification, and data append processes to enhance the client's AI algorithm accuracy and efficiency.

Input Data - Access to construction-related news articles from multi-lingual and multi-format sources.

The task was divided into two parts

• Accuracy check of auto-tagged data through verification, validation, and data append.
• Manual annotation of the 20% of data that couldn’t be auto-classified.

Accuracy Verification Process

• The data was cleaned and formatted for consistency.
• Through a robust validation and verification process, we ensured that the auto-tagged data was accurate and reliable.
• Statistical measures like precision, recall, and F1 score were used to evaluate the accuracy of auto-tagged data.
• We meticulously appended any missing details to the data, enhancing its completeness and accuracy.

Manual Text Annotation

• Developed clear and concise guidelines for annotators to handle the 20% of data that couldn't be auto-classified.
• Established KPIs, SOPs, and metrics based on a thorough project assessment and a deep understanding of the client's needs.
• A pilot phase was conducted to fine-tune the approach and identify potential bottlenecks before full-scale implementation.
• Human annotators categorized text based on predefined criteria.
• Texts, PDFs, and images were labeled by assigning metadata or tags.
  • Text Data - Reviewed and labeled text passages, marking entities and sentiments.
  • PDF Data - Extracted and tagged text and structured data like tables and charts.
  • Image Data - Labeled objects and features in images for tasks such as object detection and classification.
• Quality Control - We implemented a two-step quality check process for annotated articles, involving review and validation by experienced annotators and quality specialists to ensure precision and reliability.
• Deliverables – Classified and labeled over 10,000 construction-related articles.
• The annotated dataset was exported in a machine learning-compatible format for training and validation.

Post Project Support

• Provided ongoing monitoring and maintenance, including periodic quality assessments and updates, to sustain AI model accuracy and adapt to industry trends.
• Offered training refreshers and support to ensure the client's team remained proficient, fostering a long-term partnership for lasting results.