At LegitScript, we are passionate about making the internet and payment ecosystems safer and more transparent. We help companies of all sizes keep their services legal and safe for consumers. To do this, LegitScript combines big data with the world’s leading team of experts skilled in highly regulated and complex sectors, including transaction laundering detection, pharmaceuticals, online gambling, and more.

The result? Unmatched accuracy and deep risk analysis that identifies which commercial entities play by the rules, and which do not. Our diverse industry partnerships provide unique insights that keep businesses and governments at the forefront of emerging trends. That’s why LegitScript is trusted by the world's largest search engines, internet platforms, payment companies, and regulatory agencies.

General description

What You'll Do:

• Implement and optimize AI orchestration frameworks like crewAI to manage complex, multi-agent AI workflows and improve task delegation efficiency.
• Develop and maintain a comprehensive testing pipeline using industry-standard tools for experiment tracking, model versioning, and performance visualization.
• Design and execute A/B tests for prompt engineering, comparing different versions to maximize output quality and consistency.
• Implement advanced multi-prompt strategies including prompt ensembling, chain-of-thought prompting, and few-shot learning to enhance model performance.
• Establish best practices for prompt templating, version control, and documentation to ensure scalability and maintainability of AI systems.
• Develop a robust evaluation framework for assessing AI model outputs, including automated metrics and human-in-the-loop feedback mechanisms.
• Implement prompt chaining and reasoning techniques to break down complex tasks into manageable sub-tasks for improved accuracy and explainability.
• Create and maintain a knowledge base of effective prompting techniques, including examples and case studies, to facilitate team learning and consistency.
• Develop strategies for handling edge cases and out-of-distribution inputs, including fallback mechanisms and graceful degradation of AI system performance.
• Integrate AI systems seamlessly with the existing Node.js (TypeScript) backend and React frontend, ensuring full-stack implementation and optimization of AI features within the application architecture.

What You'll Bring:

• 5+ years of experience in software engineering, ideally within dynamic startup environments
• Programming experience with TypeScript for at least 4 years required
• Deep familiarity working with AWS
• Familiarity with Large-language models, and creating other Generative AI techniques in a SaaS environment
• Experience with COTS Generative AI solutions
• Experience with Weights and Biases platform for measuring and tuning as it relates to Generative AI
• Ability to quickly create Proof-of-Concept code and engage with multiple stake holders to drive the PoC productization
• A customer-centric approach and a relentless drive for excellence, aligned with our foundational values.
• Exceptional written and verbal communication skills, capable of inspiring and engaging others
• Self-motivation, integrity, and a track record of delivering impactful results with a high degree of autonomy
• Agility and adaptability to thrive in fast-paced work environments
• A degree in computer science or a related field, with a preference for advanced qualifications