About Prophet Security
Prophet Security is a force multiplier for security teams. Prophet leverages generative AI technology to streamline the triage, investigation, incident response, and remediation of alerts. As a result, Prophet reduces manual, tedious, repetitive work for security teams and empowers them to focus their time on higher value security tasks. We are seeking a talented software engineer to work on our efforts in developing artificial intelligence and machine learning capabilities that drive our product.
Role Summary
As our AI/ML Engineer, you'll work on the integration and optimization of large language models (LLMs) and other AI/ML technologies into our product. Your prior experience in LLM fine-tuning, evaluation, and productionization will directly enhance our product capabilities and user experience.
Key Responsibilities
LLM Integration & Feature Development: Lead the design and development of innovative features using LLMs, pushing the boundaries of Prophet's capabilities.
Prompt Engineering & Optimization: Master the art of prompt engineering, fine-tuning, and in-context learning to maximize the potential of LLMs within our product.
Model Evaluation & Benchmarking: Develop a model evaluation framework, extending benchmarks like MT-Bench to ensure the robustness and performance of our LLM implementations.
Safety & Guardrails: Establish comprehensive guardrails for safe and responsible use of LLMs, proactively mitigating potential risks.
Architectural Design: Shape the architecture of our LLM solutions, making strategic decisions on model configurability, portability, and cost (e.g., one large model vs. multiple task-specific models).
Research & Development: Stay at the forefront of LLM research, identifying groundbreaking techniques and tools to enhance our product offerings.
Required Qualifications
Experience: Proven track record of shipping LLM-powered products or features.
Technical Expertise: Deep understanding of LLMs, fine-tuning techniques, prompt engineering, model evaluation, and safety considerations. Proficient in Python.
Research Mind: Passionate about LLM research, with a demonstrated ability to translate cutting-edge, research concepts into practical applications.
Architectural Thinking: Experience designing and scaling ML systems, with a keen eye for optimizing performance and resource utilization.
In-depth Prompting Techniques: Familiarity with Chain-of-Thought (CoT), Tree of Thoughts (ToT), ReAct, and other advanced prompting strategies.
Knowledge Synthesis: Experience with Retrieval-Augmented Generation (RAG) or similar techniques.
Problem-Solving: Outstanding analytical and problem-solving skills, able to tackle complex challenges in the LLM domain.
Communication & Collaboration: Excellent communication skills to collaborate effectively with diverse stakeholders and translate technical concepts for non-technical audiences.
Bonus Qualifications
Experience building custom frameworks to streamline LLM interactions (similar to LangChain, LlamaIndex, or Guidance).
Familiarity with agentic architecture concepts and the role of LLMs in enabling agent-based systems.
Publications or Open-Source Contributions related to LLMs.
Experience with Go.