๐Ÿš€
Codex Editor
  • Project Overview
    • Welcome to Codex
    • Codex Editor
      • Features
      • What was Project Accelerate?
        • Project Philosophy
        • Steering Committee
    • Vision
      • Powerful Tools and Simplicity
      • Streamlining the Translation Process
      • Unopinionated Assistance
    • Architecture
    • Frequently Asked Questions
    • Translator Requirements
    • Roadmap
    • Why VS Code?
    • Multimodality
  • Translator's Copilot
    • What is Translator's Copilot?
    • Information Management
      • Resource Indexing
      • Greek/Hebrew Insights
      • Chat with Resources
      • Project-Based Insights
    • Translation Assistance
      • Translation Drafting
        • Prioritizing Semantics over Structures
        • Reranked Translation Suggestions
      • Quality Checking
        • Token matching evaluation
        • Few-shot evaluation
        • Fine-tuned Bible QA model
        • Character and word n-gram evaluation
        • Simulated GAN evaluation
        • Linguistic Anomaly Detection (LAD)
      • Back Translation
    • Orchestration
      • Translation Memory
      • Multi-Agent Simulations
      • Drafting linguistic resources
    • Intelligent Functions Library
  • Development
    • Codex Basics
      • Projects
      • The Editor
      • Extensions
        • Rendered Views
        • Language Servers
        • Custom Notebooks
        • Global State
    • Experimental Repositories
Powered by GitBook
On this page
  1. Translator's Copilot
  2. Translation Assistance
  3. Quality Checking

Simulated GAN evaluation

PreviousCharacter and word n-gram evaluationNextLinguistic Anomaly Detection (LAD)

Last updated 1 year ago

"Simulated GAN Evaluation" is a proposed method for assessing the quality of AI-generated translations. The process begins by prompting for a translation using 'n' examples and a directive to complete the translation pairs (see ). The entire prompt, along with the AI's completion, is then passed to the evaluator. The evaluator's task is to determine which translation was generated by the AI. This may (but hopefully it wouldn't) require fine-tuning a discriminator model on the gold-standard pairs.

The concept, inspired by Generative Adversarial Networks (GANs), involves an evaluator attempting to distinguish AI-drafted translations from a list of real translations. While it doesn't function exactly like a GAN, the idea is to use a similar principle for evaluation.

The method could potentially be used to refine the prompts or the models themselves, by replacing the evaluator with a separate AI. Initial implementation will use LLMs to gauge the viability of the concept. If successful, this approach could be extended to train GANs for specific languages.

From Ben: Idea: Have the agent pick the machine translation from either the list of reordered transitions or from two translation the real one and a machine translation. But also have it detail itโ€™s rational. You can then use the rational from the times it gets it correct to improve the the next iteration in the loop

example