#podcast-review #AI https://music.youtube.com/watch?v=x6jsU9shgFg&si=YllysFtxx7MO0V_C Sean's takeaways * Proprietary models developed by organizations with proprietary applications have a distinct advantage over those that don't because of data and the product feedback loop. The advantage goes non-linear when a model producer monetizes multiple layers of the stack. * Infra is a key building block but needs to be managed at the right layer of abstraction. Too low and power users will work around it to get the control they need. Too high and regular users will get confused by the complexity. * Model servers are critical but they are not the end-user applications, they are application support. * Fire Optimizer: SaaS Platform - takes inference workload for a partiulcar * Objective function: Speed, Quality, Cost. Spits out new model and inference configuration. Takes model as input. * 5 different ways to quantize. * Use case specific * Objective function * Lin: Doubling down on "FireOptimizer" - many customers that have viable * Going to focus on quality side -- want to maximize quality gain through customization. Should be much better quality than a generic model when you fused use-case specific data. * Data is the moat. * Comfortable with building high performance kernels for LLM, optimizing for bottlenecks, e.g. networking. Build on top of CUDA and ROCm (AMD). * Plan to expand to 10 different clouds in many regions Lin Qiao - more on her here * https://www.unite.ai/lin-qiao-ceo-co-founder-of-fireworks-ai-interview-series/ * https://thedataexchange.media/fireworks-ai-lin-qiao/ * https://music.youtube.com/watch?v=wbhGAc_itMg Can't separate training and inference systems as the model means nothing until the product says it is. There is a shift after PMF is to reduce costs and latency. Inference is the foundation and Firework's starting point. Added tuning on top of inference. Abstractions across training and inference are different. OpenAI compatible. When they graduate PMF it's about optimizing the inference stack. 3d optimization: quality, latency and cost. Optimization is a search problem with a large search space. Vertex Model Optimization announced at Google I/O Developers want to be model agnostic, want to be able to shift to SOTA. Most critical part of RFT is to write an evaluator that is a reward function. You often have to call into app specific internal API that is highly coupled with the product. Dependent on internal state. Early adopters are power users and don't want abstractions. For broader audience they want to focus on their applications. Higher level abstractions must be built on lower level abstractions. Large percentage of the data is not available on the Internet, it's inside products. Open models versus closed models. Commodity versus asset. Model quality is data quality. I don't believe any lab has proprietary or secret sauce. People move across industries and spread knowledge. Public data on Internet has been exhausted. There are 10 big data labeling companies selling to other organizations. The only difference is in the app space. The app space of data will be verticalized, unique competitive edge. How do you turn that into your own model advantage. Open models are a lot easier to tune, they have that advantage over closed models. They are betting on customizations of open models. Not just inference provider but inference tuning. Believes that closed and open models will converge because data will be exhausted. Takes a couple of hours to support a new model in Fireworks AI.