Software Engineer, Developer Experience & Evals

The Role:

We are seeking a Software Engineer, DevEx & Evals to play a highly impactful role in shaping the Fireworks platform. You will be responsible for defining and building a cohesive developer journey that bridges the gap between experimentation and production.

In the AI application lifecycle, Evals and Training is a continuous loop. You will build the experiences that allow developers to seamlessly navigate this cycle: starting with serverless experimentation, moving to fine-tuning custom models, evaluating them rigorously, and finally deploying to on-demand GPUs.

You will also lead the engineering efforts on our open-source initiative and productize these capabilities to help users author evals and train custom models. As the public platform serves as the product-led growth (PLG) engine for Fireworks, your work will directly drive business impact by removing friction at every stage of the developer onboarding and adoption process.

Key Responsibilities:

• Unify the AI Lifecycle: Build a streamlined experience that connects our serverless inference, fine-tuning, and on-demand deployment products into a single, intuitive workflow.
• Streamline Developer Onboarding: Obsess over the "Time to First Token" and "Time to First Fine-tune." Identify and eliminate friction points for new developers entering the platform.
• Architect Scalable Eval Tooling: Design full-stack features that support the continuous cycle of training and evaluation, providing deep insights into model quality that directly inform the next round of fine-tuning.
• Productize Inference Optimization: Build experiences that help developers optimize their GPU deployments for their specific workloads, guiding them on the right balance of throughput, latency, and model quality.

Minimum Requirements:

• 1 - 7 years of software engineering experience (We are hiring at multiple levels for this role).
• Passion for Developer Experience: You care deeply about API design, documentation, and the ergonomics of CLI/SDK tools.
• Understanding of the GenAI Lifecycle: You understand the end-to-end workflow—from prompting a base model to curating a dataset, fine-tuning, and productionizing agents—and how these steps interconnect.
• User-Centric Mindset: Willing to talk to users, triage GitHub issues for open-source projects, and build products from scratch to serve emerging needs.

Preferred Qualifications:

• 3+ years of software engineering experience.
• Domain-Specific Evaluation Experience: Strong familiarity with designing and running evaluations for domain-specific use cases (e.g. medical, legal, coding, or custom internal datasets)
• Open Source Contributions: Prior contributions to developer tools or AI/ML repositories
• Inference & Hardware Knowledge: Interest in the hardware side of AI—understanding GPU constraints, inference optimization techniques, and how they relate to model performance.
• Startup DNA: Experience in fast-paced environments where you own features end-to-end.