About Prophet Security

Prophet Security is a force multiplier for security teams. Prophet leverages generative AI technology to streamline the triage, investigation, incident response, and remediation of alerts. As a result, Prophet reduces manual, tedious, repetitive work for security teams and empowers them to focus their time on higher value security tasks. We are seeking a talented software engineer to work on our efforts in developing artificial intelligence and machine learning capabilities that drive our product.

Role Summary

As our AI/ML Engineer, you'll work on the integration and optimization of large language models (LLMs) and other AI/ML technologies into our product. Your prior experience in LLM fine-tuning, evaluation, and productionization will directly enhance our product capabilities and user experience.

Key Responsibilities

• LLM Integration & Feature Development: Lead the design and development of innovative features using LLMs, pushing the boundaries of Prophet's capabilities.

• Prompt Engineering & Optimization: Master the art of prompt engineering, fine-tuning, and in-context learning to maximize the potential of LLMs within our product.

• Model Evaluation & Benchmarking: Develop a model evaluation framework, extending benchmarks like MT-Bench to ensure the robustness and performance of our LLM implementations.

• Safety & Guardrails: Establish comprehensive guardrails for safe and responsible use of LLMs, proactively mitigating potential risks.

• Architectural Design: Shape the architecture of our LLM solutions, making strategic decisions on model configurability, portability, and cost (e.g., one large model vs. multiple task-specific models).

• Research & Development: Stay at the forefront of LLM research, identifying groundbreaking techniques and tools to enhance our product offerings.

Required Qualifications

• Experience: Proven track record of shipping LLM-powered products or features.

• Technical Expertise: Deep understanding of LLMs, fine-tuning techniques, prompt engineering, model evaluation, and safety considerations. Proficient in Python.

• Research Mind: Passionate about LLM research, with a demonstrated ability to translate cutting-edge, research concepts into practical applications.

• Architectural Thinking: Experience designing and scaling ML systems, with a keen eye for optimizing performance and resource utilization.

• In-depth Prompting Techniques: Familiarity with Chain-of-Thought (CoT), Tree of Thoughts (ToT), ReAct, and other advanced prompting strategies.

• Knowledge Synthesis: Experience with Retrieval-Augmented Generation (RAG) or similar techniques.

• Problem-Solving: Outstanding analytical and problem-solving skills, able to tackle complex challenges in the LLM domain.

• Communication & Collaboration: Excellent communication skills to collaborate effectively with diverse stakeholders and translate technical concepts for non-technical audiences.

Bonus Qualifications

• Experience building custom frameworks to streamline LLM interactions (similar to LangChain, LlamaIndex, or Guidance).

• Familiarity with agentic architecture concepts and the role of LLMs in enabling agent-based systems.

• Publications or Open-Source Contributions related to LLMs.

• Experience with Go.