state-of-mlops

List of awesome mlops articles.

Deploying LLMs with TorchServe + vLLM

Date: Oct 31, 2024
Tags: TorchServe, LLM
Company: PyTorch

RFP Response Generation Workflow (with Human-in-the-Loop)

Tags: Hands-on, Llama

Vector Indexes

Date: Aug 14, 2024
Tags: Learning Resource, Vector index
Company: VectorHub

BI-as-Code and the New Era of GenBI

Date: Nov 4, 2024
Tags: Blog, BI, GenAI
Company: Rill

What is Agentic RAG

Date: Nov 5, 2024
Tags: Blog, RAG Agent
Company: Weaviate

Slurm vs Kubernetes: Which to choose for your ML workloads

Date: Jun 10, 2024
Tags: Learning resource, Slurm, Kubernetes

Introducing the next-level of AI-powered workflows with Amazon Q Developer inline chat

Date: Oct 29, 2024
Tags: Amazon Q, Coding support
Company: AWS

Time-MoE

Tags: OSS, Dataset

GPT-4o Guardrails Gone: Data Poisoning & Jailbreak-Tuning

Date: Oct 30, 2024
Tags: Blog, Data poisoning
Company: FAR.AI

Evaluating Model Retraining Strategies

Date: Oct 21, 2024
Tags: Practice, Drift

Vector Databases Are the Wrong Abstraction

Date: Oct 29, 2024
Tags: Practice, OSS, pgai, pgvectorscale

Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

Date: Oct 23, 2024
Tags: PR, Calude, RPA
Company: Anthropic

Introducing quantized Llama models with increased speed and a reduced memory footprint

Date: Oct 24, 2024
Tags: OSS, Llama, Quantization
Company: Meta

Un Ministral, des Ministraux

Date: Oct 16, 2024
Tags: OSS, Mistral, Ministral
Company: Mistral

Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking

Tags: OSS, Dataset, Ranking

Deploy Llama 3.2 Vision on Amazon SageMaker

Date: Oct 17, 2024
Tags: Hands-on, SageMaker, Llama

PyTorch 2.5 Release Blog

Date: Oct 17, 2024
Tags: Release note, Pytorch

BitNet

Tags: OSS, 1-bit LLMs
Company: Microspft

Real-time Data Infrastructure at Uber

Date: Mar 31, 2021
Tags: Paper, System design
Company: Uber

Ray Batch Inference at Pinterest (Part 3)

Date: Oct 11, 2024
Tags: Ray Batch
Company: Pinterest

How Shopify improved consumer search intent with real-time ML

Date: Oct 16, 2024
Tags: Intent search, Real-time ML
Company: Shopify

Sharing new research, models, and datasets from Meta FAIR

Date: Oct 18, 2024
Tags: OSS, Data, Model, SAM
Company: Meta

STATE OF AI REPORT 2024

Tags: Report, AI

Announcing new products and features for Azure OpenAI Service including GPT-4o-Realtime-Preview with audio and speech capabilities

Date: Oct 1, 2024
Tags: Press release, Azure AI
Company: Microspft

Building a Legal AI Agent using Azure AI Search, Azure OpenAI, LlamaIndex, and CrewAI

Date: Aug 22, 2024
Tags: Blog, LLM App

How we built ngrok's data platform

Date: Sep 26, 2024
Tags: Blog, Data platform
Company: ngrok

Airbyte

Tags: OSS, Data pipeline
Company: Airbyte

Superset

Tags: OSS, BI tool
Company: Superset

How to train a model on 10k H100 GPUs?

Date: Oct 2, 2024
Tags: Blog, GPU cluster

Introducing Netflix’s Key-Value Data Abstraction Layer

Date: Sep 19, 2024
Tags: Practice, Key-Value Store, Pagination
Company: Netflix

Optimize and deploy models with Optimum-Intel and OpenVINO GenAI

Date: Sep 20, 2024
Tags: Edge, OpenVINO, LLM

GPU-Puzzles

Tags: Learning resources, GPU, CUDA

GenOps: the evolution of MLOps for gen AI

Date: Sep 21, 2024
Tags: Blog, GenOps, MLOps
Company: Google

langfun

Tags: OSS, Plug-and-play
Company: Google

Measuring Developer Goals

Date: 2024
Tags: Paper, Developer Productivity
Company: Google

GPU acceleration with Polars and NVIDIA RAPIDS

Date: Sep 17, 2024
Tags: News, Polars, GPU, NVIDIA
Company: Polars

Introducing llama-deploy, a microservice-based way to deploy LlamaIndex Workflows

Date: Sep 5, 2024
Tags: llama-deploy, workflow
Company: LlamaIndex

CUDA-Free Inference for LLMs

Date: Sep 4, 2024
Tags: Practice
Company: LlamaIndex

Learning to Reason with LLMs

Date: Sep 12, 2024
Tags: News, GPT o1
Company: OpenAI

Opik

Tags: OSS, LLMOps, Platform
Company: Comet

How Much GPU Memory is Needed to Serve a Large Language Model (LLM)?

Date: Aug 17, 2024
Tags: Tips

The Pragmatic Programmer for Machine Learning

Tags: Learning resource

GenOps: learning from the world of microservices and traditional DevOps

Date: Aug 31, 2024
Tags: Practice, GenAIOps
Company: Google

Building a serverless RAG application with LlamaIndex and Azure OpenAI

Date: Aug 27, 2024
Tags: Hands-on, RAG, LlamaIndex, Azure
Company: LlamaIndex

Building a Low-Cost Local LLM Server to Run 70 Billion Parameter Models

Date: Aug 30, 2024
Tags: Hands-on, Local, LLM
Company: comet

Things I Wished More Developers Knew About Databases

Date: Apr 22, 2020
Tags: Learning resource, DB

Aryn

Tags: OSS, Extract from PDF
Company: Aryn

Enriching and Ingesting Data into Weaviate with Aryn

Date: Sep 3, 2024
Tags: Hands-on, Aryn, Weaviate
Company: Weaviate

Locally running RAG pipeline with Verba and Llama3 with Ollama

Date: Jul 9, 2024
Tags: Hands-on, RAG, pipeline
Company: Weaviate

Prompt caching with Claude

Date: Aug 15, 2024
Tags: Prompt engineering
Company: Anthropic

Postgres as a search engine

Date: Aug 19, 2024
Tags: Blog, Postgres, Vector search

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

Date: Aug 7, 2024
Tags: Hands-on, Multimodal Language Model

Introduction to Distributed Pipeline Parallelism

Tags: Tutorial, Distributed system
Company: PyTorch

How to Deploy the Open-Source Milvus Vector Database on Amazon EKS

Date: Aug 9, 2024
Tags: Hands-on, Milvus
Company: milvus

Deploy open LLMs with Terraform and Amazon SageMaker

Date: Aug 5, 2024
Tags: Tutorial, LLMs, AWS SageMaker, Terraform

CPU-Optimized Embedding Models with fastRAG and Haystack

Date: Aug 1, 2024
Tags: Experiement, fastRAG
Company: Haystack

Securing Generative AI Deployments with NVIDIA NIM and NVIDIA NeMo Guardrails

Date: Aug 5, 2024
Tags: Tutorial, NVIDIA NIM, NVIDIA NeMo Guardrails
Company: NVIDIA

Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2

Date: Jul 25, 2024
Tags: Practice, Migration from Spark to Ray
Company: AWS

LLM Knowledge Graph Builder: From Zero to GraphRAG in Five Minutes

Date: Jun 19, 2024
Tags: Tutorial, Graph RAG
Company: neo4j

A Visual Guide to Quantization

Date: Jul 22, 2024
Tags: Learning resource, Quantization

Building A Generative AI Platform

Date: Jul 25, 2024
Tags: Practice, RAG App

What We’ve Learned From A Year of Building with LLMs

Date: Jun 8, 2024
Tags: Practice, LLM Application

Maestro: Data/ML Workflow Orchestrator at Netflix

Date: Jul 23, 2024
Tags: OSS, Workflow
Company: Netflix

Reasoning through arguments against taking AI safety seriously

Date: Jul 9, 2024
Tags: Blog, AI Safety

Introducing Micro Agent: An (Actually Reliable) AI Coding Agent

Date: Jun 19, 2024
Tags: OSS, Coding assistant
Company: builder io

LLM Evaluation doesn't need to be complicated

Date: Jul 11, 2024
Tags: Tips, LLM App Evaluation

Building and scaling Notion’s data lake

Date: Jul 1, 2024
Tags: Engineering, Data lake
Company: Notion

Deploy Multilingual LLMs with NVIDIA NIM

Date: Jul 8, 2024
Tags: Tutorial, NVIDIA NIM
Company: NVIDIA

Google Cloud TPUs made available to Hugging Face users

Date: Jul 9, 2024
Tags: NEWS
Company: HuggingFace

Extrinsic Hallucinations in LLMs

Date: Jul 7, 2024
Tags: Learning Resource, Hallucination

Multi AI Agent Systems 101

Date: Jun 17, 2024
Tags: Tutorial, CrewAI

Modernizing Uber’s Batch Data Infrastructure with Google Cloud Platform

Date: May 30, 2024
Tags: migration, GCP
Company: Uber

Step-by-Step Guide to Choosing the Best Embedding Model for Your Application

Date: Jun 4, 2024
Tags: FYI, Embeddings
Company: Weaviate

Constructing knowledge graphs from text using OpenAI functions

Date: Oct 20, 2023
Tags: Practice, Neo4j, LanguChain

MLOps Org

Tags: Org

Mastering AI Department Reorganizations: Lessons from the Trenches

Date: Jun 13, 2024
Tags: Engineering Management

Scale LLMs with PyTorch 2.0 FSDP on Amazon EKS – Part 2

Date: Apr 1, 2024
Tags: AWS EKS, LLMs Training
Company: AWS

From bare metal to a 70B model: infrastructure set-up and scripts

Date: Jun 25, 2024
Tags: Practice, ML Infra
Company: imbue

Deep dive into how Pinterest built its Text-to-SQL solution

Date: May 10, 2024
Tags: Text to SQL
Company: Pinterest

New Chunking Method for RAG-Systems

Date: Jun 2, 2024
Tags: Knowledge share

Benchmarking Haystack Pipelines for Optimal Performance

Date: Jun 24, 2024
Tags: Benchmarking, RAG, Performance
Company: Haystack

Gemma 2 is now available to researchers and developers

Date: Jun 27, 2024
Tags: OSS, LLMs
Company: Google

Data + AI Summit 2024 - Keynotes

Date: Jun, 2024
Tags: Summit, Data + AI Summit
Company: Databricks

Beyond the Basics of Retrieval for Augmenting Generation

Date: Jun 12, 2024
Tags: Practice, RAG

Sharing new research, models, and datasets from Meta FAIR

Date: Jun 18, 2024
Tags: OSS, Research, Models, Datasets
Company: Meta

How Meta trains large language models at scale

Date: Jun 12, 2024
Tags: Practice, Infrastructure
Company: Meta

Private Cloud Compute: A new frontier for AI privacy in the cloud

Date: Jun 10, 2024
Tags: Policy, Security, Privacy
Company: Apple

How to Build an End-to-End ML Pipeline in 2024

Date: Apr 7, 2024
Tags: Learning resources, MLOps

Uncensor any LLM with abliteration

Date: Jun 13, 2024
Tags: Technique, LLM Application

Advanced RAG: Corrective Retrieval Augmented Generation (CRAG) with LangGraph

Date: Apr 24, 2024
Tags: Technique, RAG Application

Monitoring LLM Security in Langfuse

Date: May 14, 2024
Tags: Tool, LLM, Security
Company: Langfuse

Let's reproduce GPT-2 (124M)

Date: Jun 10, 2024
Tags: Learning resources, Hands-on, GPT-2

Building RAG Applications with NVIDIA NIM and Haystack on K8s

Tags: Tutorial, NVIDIA NIM, Haystack

NVIDIA NIM Offers Optimized Inference Microservices for Deploying AI Models at Scale

Date: Mar 18, 2024
Tags: News, NVIDIA NIM
Company: NVIDIA

NVIDIA NIM Inference Microservice

Tags: Tutorial, NVIDIA NIM, LlamaIndex
Company: LlamaIndex

The 10 Minute Guide to Reliable RAG Systems Using Patronus AI, MongoDB Atlas, and LlamaIndex

Date: Jan 10, 2024
Tags: Tutorial, Patronus AI, RAG
Company: Patronus AI

AI in software engineering at Google: Progress and the path ahead

Date: Jun 6, 2024
Tags: Blog, Software Engineer, AI support
Company: Google

Developer Productivity for Humans, Part 7: Software Quality

Date: Dec 22, 2023
Tags: Paper, Developer Productivity
Company: Google

Open-source Model Fine-Tuning Leaderboard

Tags: Leaderboard, LLM, Fine Tuning
Company: Predibase

Hello Qwen2

Date: Jun 7, 2024
Tags: OSS, LLM
Company: Qwen

Unexpected Anti-Patterns for Engineering Leaders — Lessons From Stripe, Uber & Carta

Tags: Note, Engineering Management

From Predictive to Generative – How Michelangelo Accelerates Uber’s AI Journey

Date: May 2, 2024
Tags: System Design, MLOps, Michelangelo
Company: Uber

What We’ve Learned From A Year of Building with LLMs

Date: Jun 8, 2024
Tags: Practice, LLMs
Company: Applied LLMs

Create a Blog Writer Multi-Agent System using Crewai and Ollama

Date: May 27, 2024
Tags: Practice, LLM Apps

MLOps Infrastructure at Mission Lane (Part 1)

Date: Jan 31, 2024
Tags: System Design, MLOps
Company: Mission Lane

Powering Feature Stores with ClickHouse

Date: Jan 18, 2024
Tags: System Design, MLOps
Company: ClickHouse

How To Organize Continuous Delivery of ML/AI Systems: a 10-Stage Maturity Model

Date: May 17, 2024
Tags: Guideline, CICD, MLOps
Company: Outerbounds

Langfuse

Tags: OSS, LLMOps
Company: Langfuse

Building an Observable arXiv RAG Chatbot with LangChain, Chainlit, and Literal AI

Date: May 14, 2024
Tags: Practice, LLM App

Llama 3 implemented in pure NumPy

Date: May 16, 2024
Tags: Learning resource, Llama3, numpy

The 4 Advanced RAG Algorithms You Must Know to Implement

Date: May 4, 2024
Tags: Practice, RAG

Data Wrangler

Tags: VSCode extension, Data engineering
Company: Microsoft

MOMENT

Tags: OSS, Time series ML
Company: Carnegie Mellon University

A first attempt at DSPy Agents from scratch

Tags: Tutorial, DSPy

ScrapeGraphAI

Tags: OSS, Web Scraping, LLM

Common Pitfalls To Avoid When Using Vector Databases

Date: Apr 29, 2024
Tags: Practice, Vector database

Streaming Pipelines for Fine-tuning LLMs and RAG in Real-Time

Date: Apr 24, 2024
Tags: Streaming pipeline
Company: Comet

Building a Chat Application with LangChain, LLMs, and Streamlit for Complex SQL Database Interaction

Date: Feb 10, 2024
Tags: LangChain

A Visual Guide to Vision Transformers

Date: Apr 5, 2024
Tags: Learning resource, Vision Transformers

Snowflake Launches the World’s Best Practical Text-Embedding Model for Retrieval Use Cases

Date: Apr 16, 2024
Tags: OSS, LLM
Company: Snowflake

Rules of Machine Learning: Best Practices for ML Engineering

Tags: Practice, ML Engineering

Shepherd: How Stripe adapted Chronon to scale ML feature development

Date: Apr 15, 2024
Tags: Practice, Chronon
Company: Stripe

Verba: Building an Open Source, Modular RAG Application

Date: Mar 7, 2024
Tags: OSS, RAG Application, Verba
Company: Weaviate

Our next-generation Meta Training and Inference Accelerator

Date: Apr 10, 2024
Tags: Hardware, AI
Company: Meta

Measuring trends in AI

Tags: Report, Trend
Company: Stanford Univ

Introducing Meta Llama 3: The most capable openly available LLM to date

Date: Apr 18, 2024
Tags: LLM, Llama3
Company: Meta

NSA Publishes Guidance for Strengthening AI System Security

Date: Apr 15, 2024
Tags: Goverment, Guidance, Security
Company: NSA

Chronon, Airbnb’s ML Feature Platform, Is Now Open Source

Date: Apr 9, 2024
Tags: OSS, Feature Platform, Chronon
Company: Airbnb

But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning

Date: Apr 2, 2024
Tags: Learning Resource, Transformers

Data acquisition strategies for AI-first start-ups

Date: Apr 4, 2024
Tags: For Beginner, Data

Supporting Diverse ML Systems at Netflix

Date: Mar 8, 2024
Tags: Practice, ML system, Metaflow
Company: Netflix

Scaling AI/ML Infrastructure at Uber

Date: Mar 28, 2024
Tags: Practice, ML Infra
Company: Uber

Introducing DBRX: A New State-of-the-Art Open LLM

Date: Mar 27, 2024
Tags: OSS, LLM
Company: Databricks

CI/CD for Machine Learning in 2024: Best Practices to Build, Train, and Deploy

Date: Dec 27, 2023
Tags: Practice, MLOps

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

Date: Mar 22, 2024
Tags: LLM, Quantization
Company: Hugging Face

7 Methods to Secure LLM Apps from Prompt Injections and Jailbreaks

Date: Mar 26, 2024
Tags: Prompt Injection

Evolving from Rule-based Classifier: Machine Learning Powered Auto Remediation in Netflix Data Platform

Date: Mar 5, 2024
Tags: DevOps, Auto Remediation, OOM error
Company: Netflix

PDF-Based Question Answering with Amazon Bedrock and Haystack

Date: Jan 17, 2024
Tags: Practice, Bedrock, OpenSearch, Haystack
Company: deepset

Generative AI for Beginners

Tags: Learning resource
Company: Microsoft

Building Meta’s GenAI Infrastructure

Date: Mar 12, 2024
Tags: Infrastructure, High-Level Architecture
Company: Meta

LLM Evaluation Metrics: Everything You Need for LLM Evaluation

Date: Jan 22, 2024
Tags: deepeval, LLM Evaluation Metrics
Company: Confident AI

MIT 6.S087: Foundation Models & Generative AI (2024)

Tags: Lecture

Build an LLM-Powered API Agent for Task Execution

Date: Feb 21, 2024
Tags: LLM, Application, API Agent
Company: NVIDIA

Intro to DSPy: Goodbye Prompting, Hello Programming!

Date: Feb 29, 2024
Tags: LLMOps, DSPy, Prompt

How to Build an Advanced AI-Powered Enterprise Content Pipeline Using Mixtral 8x7B and Qdrant

Date: Feb 19, 2024
Tags: LLM, Search, Touch and Try

Top 5 Web Scraping Methods: Including Using LLMs

Date: Feb 26, 2024
Tags: Scraping
Company: comet

Deploying LLMs Into Production Using TensorRT LLM

Date: Feb 22, 2024
Tags: NVIDIA, TensorRT LLM, Tutorial

Generative AI Design Patterns: A Comprehensive Guide

Date: Feb 14, 2024
Tags: GenAI, Practice, Applicaton Pattern

Gemma Open Models

Tags: AI, OSS
Company: Google

GPT in 60 Lines of NumPy

Date: Jan 30, 2023
Tags: Learning resource

Automated Unit Test Improvement using Large Language Modelsat Meta

Tags: Paper, LLM, Unit Test Improvement
Company: Meta

My MLOps bookshelf

Date: Apr 17, 2023
Tags: Books

(Almost) Every infrastructure decision I endorse or regret after 4 years running infrastructure at a startup

Date: Feb 1, 2024
Tags: Ops, Infrastructure, Kubernetes, Good and Bad Practice

Top Evaluation Metrics for RAG Failures

Date: Feb 3, 2024
Tags: RAG, LLM, Ops, Metrics, Practice

Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices

Tags: Best Practice, Data Science, Evaluation

Enhance Conversational Agents with LangChain Memory

Date: Jan 25, 2024
Tags: LangChain, ChatOps, Chat agent
Company: commet

Doubling Down on Production AI at Tecton

Date: Dec 19, 2023
Tags: News, Feast, Tecton
Company: Tecton

LLaVA-1.6: Improved reasoning, OCR, and world knowledge

Date: Jan 30, 2024
Tags: News, LLaVA

What is the dumbest thing you have seen in data science?

Tags: reddit, dumbest, Data Science

Host the Whisper Model on Amazon SageMaker: exploring inference options

Date: Jan 16, 2024
Tags: Open AI Whisper, Amazon SageMaker
Company: AWS

Advanced RAG: Query Augmentation for Next-Level Search using LlamaIndex

Date: Jan 18, 2024
Tags: RAG, LlamaIndex

How Meta built the infrastructure for Threads

Date: Dec 19, 2023
Tags: ZippyDB, Infrastructure
Company: Meta

CI/CD for Machine Learning in 2024: Best Practices to Build, Train, and Deploy

Date: Dec 27, 2023
Tags: CI/CD/CT, MLOps, ML Lifesycle

https://github.com/collabora/WhisperSpeech

Tags: OSS, Speech to Text

https://github.com/recommenders-team/recommenders

Tags: OSS, Recommender system, Algorithm

Generating value from enterprise data: Best practices for Text2SQL and generative AI

Date: Jan 4, 2024
Tags: LLM application, AWS Bedrock
Company: AWS

Merge Large Language Models with mergekit

Date: Jan 9, 2024
Tags: OSS, LLMs merge, Algorithm

https://github.com/DAGWorks-Inc/hamilton

Tags: OSS, DAG, Dataflow

2023: A year of groundbreaking advances in AI and computing

Date: Dec 22, 2023
Tags: Summary
Company: Google

Evaluating Prompts: A Developer’s Guide

Date: Dec 19, 2023
Tags: LLM, Prompt engineering, Evaluation guide
Company: arize

Speculative Decoding for 2x Faster Whisper Inference

Date: Dec 20, 2023
Tags: Whisper, Engineering

StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation

Tags: Diffusion model, OSS

How To Use Comet At Different Stages of ML Projects

Date: Dec 20, 2023
Tags: Practice, MLOps
Company: Comet

Anti-hype LLM reading list

Tags: LLM links

How Pinterest scaled to 11 million users with only 6 engineers

Date: Oct 3, 2023
Tags: Engineering, Practice
Company: Pinterest

Optimizing LLMs From a Dataset Perspective

Date: Sep 15, 2023
Tags: LLMs, Dataset, Finetuning

Advanced RAG Techniques: an Illustrated Overview

Date: Dec 17, 2023
Tags: RAG, Techniques

Efficient Vector Similarity Search in Recommender Workflows Using Milvus with NVIDIA Merlin

Date: Dec 15, 2023
Tags: Practice, Vector search, Recommender, NVIDIA Merlin, Milvus
Company: Milvus

Our First Netflix Data Engineering Summit

Date: Dec 15, 2023
Tags: Data Engineering, Practice
Company: Netflix

Minimize real-time inference latency by using Amazon SageMaker routing strategies

Date: Nov 30, 2023
Tags: SageMaker, real-time inference
Company: AWS

A Guide on 12 Tuning Strategies for Production-Ready RAG Applications

Date: Dec 6, 2023
Tags: RAG, Practice

Deploy Mixtral 8x7B on Amazon SageMaker

Date: Dec 12, 2023
Tags: RAG, SageMaker, Mixtral

データ品質の5つの分類と品質管理プロセス

Date: Dec 18, 2023
Tags: Data engineering
Company: 風音屋

Extracting Training Data from ChatGPT

Date: Nov 28, 2023
Tags: LLMs, Vulnerabilities

LMQL — SQL for Language Models

Date: Nov 28, 2023
Tags: LLMOps

Designing a Distributed SQL Engine: Challenges & Decisions

Date: Apr 23, 2023
Tags: Engineering
Company: OceanBase

Unveiling the Core of Instacart’s Griffin 2.0: A Deep Dive into the Machine Learning Training Platform

Date: Nov 21, 2023
Tags: ML Platform, Training
Company: Instacart

Mastering ML Model Evaluation with Giskard: From Validation to CI/CD Integration

Date: Oct 24, 2023
Tags: Model evaluation, Scan, Vulnerability
Company: Giskard

Boost inference performance for LLMs with new Amazon SageMaker containers

Date: Nov 27, 2023
Tags: LLMs, AWS SageMaker
Company: AWS

Automatic detection of hallucination with SelfCheckGPT

Date: Nov, 2023
Tags: Hallucination

Learning JAX as a PyTorch developer

Date: Nov 9, 2023
Tags: JAX, PyTorch, Tips

make real, the story so far

Date: Nov 19, 2023
Tags: Application, OpenAI, ChatGPT

Ingesting Data for Semantic Searches in a Production-Ready Way

Date: Nov 19, 2023
Tags: LLMOps

Key takeaways from the Biden administration executive order on AI

Date: Oct 31, 2023
Tags: Executive Order on AI

High-Performance Llama 2 Training and Inference with PyTorch/XLA on Cloud TPUs

Date: Nov 6, 2023
Tags: TPU, PyTorch, Llama2, Training, Inference, High performance
Company: PyTorch

Boosting RAG: Picking the Best Embedding & Reranker models

Date: Nov 3, 2023
Tags: RAG, Evaluation, Rerankers

Production-Ready Observability Platform for AI Systems

Date: Nov 3, 2023
Tags: MLOps, Observability, Good Reference

New models and developer products announced at DevDay

Date: Nov 6, 2023
Tags: Update, ChatGPT
Company: OpenAI

Lessons Learned from Twenty Years of Site Reliability Engineering

Date: Oct, 2023
Tags: SRE, Lessons learned
Company: Google

The Guide To LLM Evals: How To Build and Benchmark Your Evals

Date: Oct 13, 2023
Tags: LLMOps, Performance evaluation

An Intro to Real-Time Machine Learning

Date: Oct, 2023
Tags: Feature store, Realtime ML

Branches Are All You Need: Our Opinionated ML Versioning Framework

Date: Oct 10, 2023
Tags: ML versioning, Git branch

Advanced RAG Implementation on Custom Data Using Hybrid Search, Embed Caching And Mistral-AI

Date: Oct 9, 2023
Tags: RAG, Implementation

Reflections on AI Engineer Summit 2023

Date: Oct, 2023
Tags: LLMOps, Practice

Startup CTO Handbook

Tags: Engineering management

State of AI Report 2023

Date: Oct 12, 2023
Tags: Report

Analyzing the Security of Machine Learning Research Code

Date: Oct 4, 2023
Tags: Security, Tips, Practice, Credential
Company: NVIDIA

Train and Deploy Mistral 7B with Hugging Face on Amazon SageMaker

Date: Oct 5, 2023
Tags: AWS SageMaker, Serving an LLM, WebAPI

Personalize your generative AI applications with Amazon SageMaker Feature Store

Date: Oct 6, 2023
Tags: AWS SageMaker, Serving an LLM
Company: AWS

Multimodality and Large Multimodal Models (LMMs)

Date: Oct 10, 2023
Tags: LMM, Multimodal, CLIP, Flamingo

Scaling Large (Language) Models with PyTorch Lightning

Date: Oct 4, 2023
Tags: PyTorch Lightning, Training, Sample
Company: Lightning AI

Retrieval Augmented Generation on audio data with LangChain and Chroma

Date: Sep 26, 2023
Tags: RAG
Company: Assembly AI

Chroma

Tags: OSS, Vector database

Weaviate

Tags: OSS, Vector database

Prompt Engineering Evolution: Defining the New Program Simulation Prompt Framework

Date: Sep 29, 2023
Tags: Prompt engineering

Training Foundation Improvements for Closeup Recommendation Ranker

Date: Sep 27, 2023
Tags: MLOps, Automation
Company: Pinterest

How LinkedIn Is Using Embeddings to Up Its Match Game for Job Seekers

Date: Oct 5, 2023
Tags: Embeddings, Two tower, Recommender system
Company: LinkedIn

Lessons learned from implementing user-facing analytics / dashboards?

Date: Sep 29, 2023
Tags: Lessons learned, BI tool

10 Ways to Improve the Performance of Retrieval Augmented Generation Systems

Date: Sep 19, 2023
Tags: RAG

Causality for Machine Learning

Tags: Causality, Book

7 Habits of Highly Effective Software Engineers

Tags: Mindset

Accelerating Vector Search: Fine-Tuning GPU Index Algorithms

Date: Sep 11, 2023
Tags: Vector search, Benchmark, RAPIDS AI RAFT
Company: NVIDIA

LLM Monitoring and Observability — A Summary of Techniques and Approaches for Responsible AI

Date: Sep 15, 2023
Tags: LLMOps, Monitoring, Evaluating, Tracking

Google’s Bard chatbot can now find answers in your Gmail, Docs, Drive

Date: Sep 19, 2023
Tags: Bard, Chatbot, Application
Company: Google

DALL·E 3

Tags: Text-to-image, ChatGPT plus
Company: OpenAI

Evaluation & Hallucination Detection for Abstractive Summaries

Date: Sep, 2023
Tags: Hallucination, survey, LLM, Summarization

Optimizing LLMs From a Dataset Perspective

Date: Sep 15, 2023
Tags: LLM, Fine tuning, Technique

LLM Training: RLHF and Its Alternatives

Date: Sep 10, 2023
Tags: LLM, Tuning, Training model, RLHF

Unlocking Multi-GPU Model Training with Dask XGBoost

Date: Sep 7, 2023
Tags: XGBoost, RAPIDS, Dask, Multi-GPU
Company: NVIDIA

Introducing Glassdoor’s ML Registry: A Centralized Artifact Management Solution

Date: Aug 31, 2023
Tags: Artifact registory, In-house solution
Company: Glassdoor

Automated trace collection and analysis

Date: Sep 5, 2023
Tags: CPU, GPU, Utilization, Profiling
Company: PyTorch

Optimize open LLMs using GPTQ and Hugging Face Optimum

Date: Aug 31, 2023
Tags: LLMs quantization, GPTQ

Organize Your Prompt Engineering with CometLLM

Date: Aug 26, 2023
Tags: CometLLM
Company: Comet

Accelerating AI: Implementing Multi-GPU Distributed Training for Personalized Recommendations

Date: Jun 8, 2023
Tags: Distributed training, DP (Data Parallel), DDP (Distributed Data Parallel)

Evaluating the fairness of computer vision models

Date: Aug 31, 2023
Tags: DINOv2, FACET, dataset, fairness, CV
Company: Meta

Teaching with AI

Date: Aug 31, 2023
Tags: GPT, LLMs, Prompt
Company: OpenAI

Lessons Learnt From Consolidating ML Models in a Large Scale Recommendation System

Date: Aug 25, 2023
Tags: Architect, ML system
Company: Netflix

Retrieval-Augmented Generation: How to Use Your Data to Guide LLMs

Date: Aug 24, 2023
Tags: LLMs, RAG, Tunining, Vector search, Guide

Measuring developer productivity? A response to McKinsey

Date: Aug 30, 2023
Tags: Note, Productivity, DORA, SPACE

Cerebrium

Tags: SaaS, Serverless
Company: cerebrium

An Elegant Puzzle: Systems of Engineering Management

Tags: Engineering management, Reading note

Introducing Code Llama, a state-of-the-art large language model for coding

Date: Aug 24, 2023
Tags: Coding support, OSS
Company: Meta

How to Build a Fully Automated Data Drift Detection Pipeline

Date: Aug 2, 2023
Tags: Drift detection, Workflow, Evidently, Kestra

ML Pipeline Architecture Design Patterns (With 10 Real-World Examples)

Date: Aug 11, 2023
Tags: Real example, ML pipeline, Workflow
Company: Neptune

Deploy thousands of model ensembles with Amazon SageMaker multi-model endpoints on GPU to minimize your hosting costs

Date: Aug 8, 2023
Tags: SageMaker, Multi-Model Endpoints, MMEs
Company: Amazon

Curating Trillion-Token Datasets: Introducing NVIDIA NeMo Data Curator

Date: Aug 8, 2023
Tags: NeMo, Data curation, Data scraping
Company: NVIDIA

A layered approach to MLOps

Date: Dec 20, 2022
Tags: MLOps, DDD, Working with researcher and engineer, Practice
Company: Microsoft

Tutorial: Build an Active Learning Pipeline using Data Engine

Date: Aug 15, 2023
Tags: Data Engine
Company: DagsHub

Patterns for Building LLM-based Systems & Products

Date: Jul, 2023
Tags: LLMs, Evaluation, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect user feedback

Effectively solve distributed training convergence issues with Amazon SageMaker Hyperband Automatic Model Tuning

Date: Jun 13, 2023
Tags: Hyperparameter, tuning, distributed training
Company: Amazon

Launching Data Engine – A toolset for rapid iteration on unstructured datasets

Date: Jun 24, 2023
Tags: Dataset, Data engine, Data engineering
Company: DagsHub

Adding Interpretability to PyTorch Models with Captum

Date: Jun 28, 2023
Tags: Explainability

Explainable AI: Visualizing Attention in Transformers

Date: Jul 18, 2023
Tags: Explainability, Transformers, Attention

ML system design: 200 case studies to learn from

Date: Jun 12, 2023
Tags: List of ML system design
Company: Evidently AI

How to Deploy an AI Model in Python with PyTriton

Date: Jun 28, 2023
Tags: PyTriton, Triton Inference Server
Company: NVIDIA

Dealing with Train-serve Skew in Real-time ML Models: A Short Guide

Date: Jun 23, 2023
Tags: Drift detection, Realtime ML, Monitoring
Company: Nubank

You Don't Need a Bigger Boat

Tags: MLOps, Sample

MLOps Landscape in 2023: Top Tools and Platforms

Tags: Landscape
Company: Neptune AI

NeMo Guardrails

Tags: Chatbot, LLMs, Guardrails, OSS
Company: NVIDIA

Mitigations

Tags: Security
Company: Mitre

8 annoying A/B testing mistakes every engineer should know

Tags: Tips, AB test, mistake

Monitoring Machine Learning Models in Production

Date: Jun 13, 2023
Tags: Monitoring
Company: Comet

Arize-ai/phoenix

Date: Jun 19, 2023
Tags: Visualization, Notebook, Drift detection

How Google Measures and Manages Tech Debt

Date: Jun 9, 2023
Tags: Technical debt, Maturity model
Company: Google

Model CI/CD with Comet

Date: May 25, 2023
Tags: MLOps, Automation, CICD, Drift detection
Company: Comet

ML Ops at Reasonable Scale

Date: Jul 23, 2021
Tags: Pipeline, MLOps, Data engineering

OWASP Top 10 List for Large Language Models

Date: Jun 12, 2023
Tags: Security, Generative AI, Checklist

Securing AI Systems — Defensive Strategies

Date: Jun 7, 2023
Tags: Security, Generative AI, Strategy

Large Language Models: A Complete Guide

Date: May 31, 2023
Tags: LLMs, Training, Deploying, Various topics

Efficiently Scale LLM Training Across a Large GPU Cluster with Alpa and Ray

Date: May 15, 2023
Tags: Alpa, Ray, JAX, Building LLMs
Company: NVIDIA

DevEx: What Actually Drives Productivity

Date: May 3, 2023
Tags: DevEx, Productivity, Management

MLOps Guide

Tags: MLOps, Reference, Career

Scaling deep retrieval with TensorFlow Recommenders and Vertex AI Matching Engine

Date: May 2, 2023
Tags: MLOps, Retrieval, TwoTower, GCP
Company: Google

Recommended Resources for Starting A/B Testing

Date: Apr 20, 2023
Tags: A/B Testing, Links, Reference

Evaluate Your Team’s ML Maturity

Date: Apr 12, 2023
Tags: MLOps, Reproducibility, Visibility, Debugging, Monitoring
Company: Comet

A Practitioner's Guide to Monitoring Machine Learning Applications

Date: Apr 11, 2023
Tags: Observability

Deploy ML models at the edge with Microk8s, Seldon and Istio

Date: Mar 24, 2023
Tags: Edge, IoT, Tips

Building a Machine Learning Platform [Definitive Guide]

Date: Mar 23, 2023
Tags: MLOps, Platform
Company: Neptune AI

The Magic of Merlin: Shopify's New Machine Learning Platform

Date: Apr 6, 2022
Tags: Ray, Merlin, MLOps, Platform
Company: Shopify

Training 175B Parameter Language Models at 1000 GPU scale with Alpa and Ray

Date: Mar 22, 2023
Tags: LLMs, Alpa, Ray, Training
Company: anyscale

HOW DISCORD STORES TRILLIONS OF MESSAGES

Date: Mar 6, 2023
Tags: MogoDB, Cassanra, ScyllaDB, journey
Company: Discord

Distributed Training: Errors to Avoid

Date: Mar 2, 2023
Tags: Tips, Distributed training
Company: Neptune AI

SOON (Spark cOntinuOus iNgestion) for near real-time data at Coinbase - Part 1

Date: Jan 25, 2023
Tags: Real-time data processing
Company: Coinbase

NVIDIA Merlin meets the MLOps ecosystem: building a production-ready RecSys pipeline on cloud

Date: Feb 23, 2023
Tags: Recommender, MLOps, OSS
Company: NVIDIA

Unleashing ML Innovation at Spotify with Ray

Date: Feb 1, 2023
Tags: Ray
Company: Spotify

Curated papers, articles, and blogs on data science & machine learning in production.

Date: Feb 27, 2023
Tags: Curated list, MLOps, Production

Blueprints for recommender system architectures: 10th anniversary edition

Date: Jan 29, 2023
Tags: Recommender

Global expansion of Machine Learning Models: how to distribute them as software products?

Date: Jan 13, 2023
Tags: Globalization, Lessons learned
Company: Nubank

FastAPI Best Practices

Date: Jan 10, 2023
Tags: FastAPI, Best practice

Measuring an engineering organization

Date: Jan 2, 2023
Tags: Engineering management, Measuring productivity

Bringing Machine Learning to Production at Ubisoft

Date: Dec 29, 2022
Tags: Usecase
Company: Ubisoft

Serve hundreds to thousands of ML models — architectures from industry

Date: Jan 14, 2022
Tags: Serving, Inferencing, Pattern

Using MLOps to Build a Real-time End-to-End Machine Learning Pipeline

Date: Nov 29, 2022
Tags: MLOps, Use case, pipeline
Company: Binance

Securing Machine Learning Algorithms

Date: Dec 14, 2021
Tags: Security, Reference

Thoughts on ML Engineering After a Year of my PhD

Date: Jul 18, 2022
Tags: Consideration, ML Engineering

Argo Rollouts at scale: Bringing Automated Rollbacks to 2,100+ services at Monzo

Date: Nov 7, 2022
Tags: Argo rollouts, experience, automation
Company: Monzo

The Architecture of a Modern Startup

Date: Nov 5, 2022
Tags: Use case
Company: SYZYGY AI

The Growing Importance of Metadata Management Systems

Date: Oct 27, 2022
Tags: Metadata management, Introduction

DATA MESH ARCHITECTURE

Date: Oct 27, 2022
Tags: Data mesh, Introduction

Lessons Learned: The Journey to Real-Time Machine Learning at Instacart

Date: Sep 7, 2022
Tags: Real time ML, Use case
Company: Instacart

Principles for the security of machine learning

Date: Aug 31, 2022
Tags: Security, Principles

Best Practices for ML Engineering

Date: Sep 12, 2022
Tags: MLOps, Principles
Company: Google

ML Education at Uber: Frameworks Inspired by Engineering Principles

Date: Jul 28, 2022
Tags: MLOps, Reference, Guide
Company: Uber

Towards data quality management at LinkedIn

Date: Jun 9, 2022
Tags: Data, Architecture, Design
Company: LinkedIn

Managing Uber’s Data Workflows at Scale

Date: Feb 28, 2019
Tags: Data, ETL, Architecture, Design
Company: Uber

Supercharging A/B Testing at Uber

Date: Jul 21, 2022
Tags: AB testing, Architecture, Design
Company: Uber

A Guide to Data Annotation and Synthetic Data Generation Tools

Date: Jul 12, 2022
Tags: Annotation, Syntetic data, Introduction

Scaling productivity on microservices at Lyft (Part 1)

Date: Nov 11, 2021
Tags: Kubernetes, Productivity, Use case, Engineering
Company: Lyft

How Netflix Scales its API with GraphQL Federation (Part 1)

Date: Nov 10, 2020
Tags: GraphQL, Use case, Engineering
Company: Netflix

Operating Apache Pinot @ Uber Scale

Date: Oct 20, 2020
Tags: Pinot, Use case, Engineering
Company: Uber

Stanford CS25: Transformers United

Date: Jul 19, 2022
Tags: ML, Learning-resource, Transformers
Company: Stanford

Deployment for Free -- A Machine Learning Platform for Stitch Fix's Data Scientists

Date: Jul 19, 2022
Tags: MLOps, In-House case, Stitch Fix
Company: Stitch

Machine Learning Operations (MLOps): Overview, Definition, and Architecture

Date: Jul 11, 2022
Tags: MLOps, Definition, Role and responsibility

Shipping to Production

Date: Jul 4, 2022
Tags: MLOps, Production, Introduction

MLOps: A Taxonomy and a Methodology

Date: Jul 4, 2022
Tags: MLOps, Overview, Survey

Evidently

Date: Jul 4, 2022
Tags: MLOps, Monitoring, Drift detection
Company: Evidently AI

Made With ML #MLOps

Date: Jun 27, 2022
Tags: MLOps, Tutorial, Reference

The SPACE of Developer Productivity

Date: Jun 13, 2022
Tags: Productivity, Engineering Management

My MLOps Stack

Date: May 30, 2022
Tags: MLOps, Reference

Monzo’s machine learning stack

Date: May 2, 2022
Tags: MLOps, Operations
Company: Monzo

Gitlab DevSecOps 2021 Survey

Date: Apr 25, 2022
Tags: DevSecOps, Survey
Company: Gitlab

Interpretable Machine Learning

Date: Mar 29, 2022
Tags: Guide, Explainablity

Scaling Machine Learning Productivity at LinkedIn

Date: Jan 3, 2019
Tags: Auto ML, Feature store, Monitoring
Company: LinkedIn

The journey to build an explainable AI-driven recommendation system to help scale sales efficiency across LinkedIn

Date: Apr 6, 2022
Tags: Explainability, Recommendation, Narrative generation
Company: LinkedIn

Introducing Fabricator: A Declarative Feature Engineering Framework

Date: Apr 4, 2022
Tags: Data, Data Engineering, Feature Store
Company: DoorDash

Rules of Machine Learning

Tags: General

Data Distribution Shifts and Monitoring

Date: Feb 7, 2022
Tags: General, Drift, Data Distribution, Observability

Scaling Kubernetes to 7,500 Nodes

Date: Jan 25, 2021
Tags: Kubernetes, Modeling platform, Computing resource management

Explaination of MLOps and its common terms

Date: Sep 29, 2022
Tags: General, Drift, Data Distribution, Observability
Company: Censius

Curated list and description of MLOps tools

Date: Sep 29, 2022
Tags: General, Drift, Data Distribution, Observability , Guide
Company: Censius