Rigorous Multi-Stage Validation for Optimal Quality,
Accuracy, and Consistency in AI Model Training Data
Worker Training and Selection To meet client quality standards and deadlines, we conduct one-on-one training and testing. Only workers who have fully grasped the guidelines are selected to process the data.
01 Self-Review After completing their work, each worker performs an initial self-review to ensure they've accurately understood and applied the guidelines and to check for basic errors.
02 Cross-Review For specific projects, other skilled workers or peers of the same qualification conduct a cross-review of the work. This helps identify errors from an objective perspective and improves labeling consistency.
01 Dedicated Reviewer Assignment Each project is assigned a professional reviewer with extensive knowledge and labeling experience relevant to that domain.
02 Sampling Inspection & Full Inspection Depending on the project's scale and importance, either a full inspection of all data or a meticulous examination of representative samples is conducted. For complex or highly critical data, a 100% full inspection is the standard.
01 Accuracy We measure the percentage to which the labeled data matches the correct answers.
02 Consistency To prevent individual subjectivity, we measure how consistent the results are when different workers label the same object.
03 Completeness We verify that all necessary objects or information have been labeled without omission.
04 Validity We confirm that the labeled data aligns with the project's purpose and requirements and does not violate established rules.
05 Model Suitability Verification This is the final step to confirm if the labeled data is practically useful for AI model training. Before final delivery, a small portion of the labeled data is actually used to train the AI model, and its initial performance is reviewed.
01 Communication with Clients We initially deliver the validated data to our clients and actively gather their feedback.
02 Integration & Re-validation Based on client feedback, we perform additional labeling or modifications if necessary. The data then undergoes a re-validation process to finalize it.
03 Version Control We systematically manage all modification and update histories to ensure data transparency and traceability.
Requirement Analysis Analyze client requirements Define necessary worker qualifications
Job Posting Our network includes over 5,000 registered global freelancers. Team leaders for each language recruit team members Post job advertisements on various social media platforms → Submit basic information and required documents (resume, employment certificates, educational certificates, etc.)
Review Select workers that match the client's requirements
Identity Verification Check restricted areas → If from a restricted area, verify current residence documents (Passport or Residence Permit or Bank Book, etc.) Check for international threats → Suspected individuals → Verify through Interpol search
Career Verification Verify certificates submitted by the worker
Training and Orientation Provide initial training and process orientation
Trial Task Assignment Assign trial tasks and monitor performance
Performance Evaluation Determine final approval based on trial task performance
Contract Preparation Confidentiality regarding the project Minimum daily working hours when assigned Task costs and settlement
Formal Assignment Assign to official projects and conduct ongoing performance management