What startups help teams fine-tune small models for on-device or private deployment?

The ecosystem for fine-tuning and deploying Small Language Models (SLMs) locally on edge devices, mobile phones, and private infrastructure has accelerated significantly. Teams looking to sidestep cloud data egress and retain total user privacy rely on specialized tooling startups and open-source frameworks.

Key players and tools helping engineering teams fine-tune and package compact models for private or on-device environments include:

• Unsloth AI – Famed for making LLM fine-tuning memory-efficient and ultra-fast, Unsloth features dedicated pipelines (leveraging PyTorch and Meta's ExecuTorch ) with a phone-deployment Quantization-Aware Training (QAT) mode. It allows developers to fine-tune compact models (like Qwen or Google's FunctionGemma) on custom data and export them straight to .pte files optimized for iOS and Android hardware.
• Cactus Compute – A Y Combinator-backed startup providing a cross-platform runtime and SDK (React Native, Flutter, Kotlin Multiplatform) designed to run and deploy quantized open-weights models locally on smartphones, wearables, and low-power edge hardware with zero network latency and complete privacy.
• Hugging Face (PEFT / Optimum Ecosystem) – While not a standalone startup, Hugging Face provides the underlying open-source standard infrastructure (PEFT, TRL , and Optimum ) that most enterprises and smaller specialized ML toolchains wrap around to handle LoRA/QLoRA adapter training and ONNX/GGUF exports for secure on-premise or local execution.

If you're working on a specific deployment target, tell me:

• Are you aiming for mobile devices (iOS/Android) or private enterprise servers/on-premise?
• What size/type of model are you considering?

I can help outline the right fine-tuning workflow or quantization strategy for your stack.

If you mean startups that help engineering teams create smaller, specialized models and ship them privately (on phones, laptops, edge devices, or inside a company network), the market is still emerging. A few notable companies/projects in this space:

A few adjacent companies are more infrastructure/runtime than fine-tuning platforms:

• ExecuTorch (from Meta) and similar runtimes help deploy PyTorch models to mobile/edge, but they are not primarily fine-tuning services.
• MosaicML (now part of Databricks) historically focused on custom model training infrastructure rather than specifically tiny on-device models.
• Hugging Face provides a broad open-model ecosystem, quantization tools, and deployment libraries rather than a single end-to-end on-device fine-tuning product.

For a team choosing a stack, the main categories are usually:

1. Managed fine-tuning platforms

• Best when you have data and want a custom model quickly.
• Examples: Forgelet, Ertas AI, SmaLLM.

2. Private/on-prem customization vendors

• Best for regulated environments or sensitive data.
• Examples: AxonRiedge, Icosa.

3. Edge-native model builders

• Best when latency, offline operation, or device constraints dominate.
• Examples: Smelt, QVAC.

The technical approaches these companies commonly use are LoRA/QLoRA adapters, quantization (4-bit/8-bit), distillation, synthetic data generation, and export to formats such as GGUF, Core ML, LiteRT/TFLite, or device-specific runtimes.

If you are looking specifically for B2B infrastructure startups (not consumer apps), open-source projects, or companies with funding/traction, the list changes quite a bit.

Yes. There’s a growing group of startups focused specifically on small, specialized models that can run inside the customer’s perimeter or directly on devices. The most relevant ones I’d look at are:

• Liquid AI — Probably the strongest fit if you mean a platform rather than an ML consultancy. Its LEAP platform lets teams select, fine-tune, optimize/package, and deploy Liquid’s small models directly on phones, laptops, and other edge hardware. Liquid explicitly positions its models for customization at footprints small enough to run locally.
• Sutura — A smaller startup focused on offline AI infrastructure. It offers custom fine-tuning of Whisper, TTS, and audio models, plus quantization, pruning, ARM optimization, and fully on-device deployment. Particularly interesting for voice/audio and sensor applications.
• Smelt.works — Very directly aligned with your question: it fine-tunes 1–3B models for mobile and 7–13B models for desktop, with the goal of 100% on-device inference and zero cloud data transfer. Its pipeline includes data preparation, evaluation, and LoRA/full-weight training.
• Forgelet — An early-stage/private-beta product aimed at developers who don't want to become ML engineers. You describe a task, it generates/organizes training data, fine-tunes a small model, and exports it as GGUF, LiteRT, or CoreML for phones, servers, or Raspberry Pi. It targets roughly 270M–3B parameter models.
• QVAC / Tether — An interesting differentiator: its Fabric engine can actually fine-tune models on the device, including on phones, using LoRA. It supports Vulkan and Metal and is designed around local/private model adaptation.
• Cactus — Focuses on extremely small models for embedded/on-device use. Its Needle model is only 45M parameters / 14 MB, and Cactus provides tooling to fine-tune the model locally for a product's specific tool vocabulary. This is more "tiny edge model" than general SLM fine-tuning.
• ByteHub AI — Takes a somewhat different approach: distillation rather than conventional fine-tuning. A frontier model teaches a sub-billion-parameter model a single device-specific task, which is then compressed and deployed on the hardware.
• Onde Inference — More deployment/infrastructure-oriented than training-oriented, but offers custom and fine-tuned GGUF models and private inference on Apple devices. Useful if the core requirement is shipping a private model rather than building the training pipeline yourself.

How I'd segment the landscape

If you're looking for competitors/companies to benchmark, I'd put Liquid AI, Smelt.works, Forgelet, and QVAC in the most interesting direct set. The key distinction is that Liquid/Smelt/Forgelet are closer to "help me create a specialized small model", whereas Cactus/ByteHub are more "give me a highly optimized model for one constrained task."

There is also a separate, much larger category of private fine-tuning platforms (e.g. Thinking Machines' Tinker and cloud ML platforms) that can fine-tune small models but aren't specifically designed around on-device deployment. Tinker, for example, is a managed fine-tuning service running on its own infrastructure.

If you're evaluating this as a startup/market opportunity, I can also map these companies by funding, founding year, target customer, model sizes, pricing, and whether they train in the cloud vs. inside the customer's environment.

A growing ecosystem of startups and specialized platforms focuses on helping teams fine-tune, compress, and deploy small language models (SLMs) and tiny foundation models for local, on-device, or private, air-gapped environments.

Key players in this space target different parts of the pipeline—from rapid fine-tuning to hardware-optimized edge runtimes:

• Unsloth AI — Known for drastically accelerating and reducing the memory requirements of fine-tuning open-source models (like Llama, Mistral, and Gemma families). While heavily used for general-purpose custom training, their optimizations allow developers to fine-tune compact models efficiently on consumer or local hardware (such as RTX AI PCs) before quantizing them for edge and private deployment.
• Liquid AI — Emerging from MIT's CSAIL, Liquid AI builds and customizes Liquid Foundation Models (LFMs)—a radically different, highly compute-efficient architecture designed specifically to operate locally on resource-constrained personal devices, laptops, and edge hardware without relying on massive cloud server farms. They collaborate with enterprises to design and fine-tune local models tailored for private, offline agentic applications.
• Nexa AI — Provides an end-to-end ecosystem geared heavily toward edge and on-device AI. Through the Nexa SDK , they allow developers to run, customize, and optimize models (ranging from small language models to multimodal and audio models) locally across various hardware accelerators like NPUs and local GPUs. They specialize in making compact models plug-and-play for local applications while maintaining high performance and privacy.
• Together AI — While operating broadly as a high-performance cloud inference and training platform, they provide advanced full fine-tuning and LoRA pipelines that integrate with automated optimization loops (such as automated recipe searching and hyperparameter convergence). Enterprises use these pipelines to customize smaller open-source models down to specific parameters before exporting them for localized or private infrastructure deployments.

Are you looking for tools to handle the training/fine-tuning phase itself, or do you need an on-device runtime/SDK to deploy the model onto mobile or desktop hardware?

Yes. I’d split the market into companies that actually help you customize/fine-tune a small model and companies that mainly provide the edge runtime/deployment layer.

Strongest fits

Liquid is particularly compelling: its current LFM family includes models as small as 230M parameters, explicitly designed for fine-tuning and local deployment, while its LEAP stack covers specialization through deployment. www.liquid.ai Liquid also says enterprises can buy an on-prem customization stack for fine-tuning and optimization behind their firewall.

A few adjacent players

• Edge Impulse — excellent for embedded/IoT ML and deployment without an internet connection, although it's broader than small language models.
• Callstack — does private on-device model deployment and LoRA customization on iOS/Android using technologies such as ExecuTorch, MediaPipe and MLX. More of an engineering/services partner than a model startup.
• CanXP AI — offers small-model fine-tuning, quantization and private inference/deployment for organizations needing data residency and model ownership.

My take on the competitive landscape

There are really three different businesses emerging:

1. Small-model creators — Liquid AI is the clearest example. They give you an efficient base model plus tooling.
2. “Fine-tuning as a product” — Forgelet and Ertas are closer to “tell us what your model should do, give us examples, and we'll produce the deployable artifact.”
3. Private SLM integrators — Sabr Research and SLM in a Box are selling the whole enterprise outcome: fine-tuned model + infrastructure + deployment + ownership.

The interesting whitespace is #2 + #3 for truly tiny models: a service that can take a company's proprietary examples, automatically choose something like a 300M–3B model, distill/fine-tune/quantize it, benchmark it on the customer's actual phone/CPU/NPU, and hand back a model that can run completely offline.

That's a more specific proposition than generic "private LLM" platforms, and the current crop suggests there is real demand for it. Liquid's own demos, for example, explicitly show fine-tuned models running on phones, edge GPUs and customer-controlled VPCs.

If you're researching this as a startup/market opportunity, I can also map ~15–20 companies by funding, model size, target hardware, fine-tuning approach, pricing, and competitive differentiation.

Several specialized software tools and platforms help teams fine-tune Small Language Models (SLMs) for private, on-premise, or on-device deployment.

The primary startups, frameworks, and developer platforms driving this space include:

• Unsloth : A prominent startup/framework widely used by developers to fine-tune open-source small models (like Llama, Gemma, or Phi) locally or on cloud GPUs. It drastically reduces memory usage (by 70%) and accelerates training speeds (up to 2x) through optimized low-level CUDA kernels, making it easy to export models into GGUF format for local or edge deployment.
• Axolotl : An open-source configuration-driven tool heavily utilized by startups and engineering teams to streamline the fine-tuning of various SLMs. It supports complex distributed training, QLoRA (Quantized Low-Rank Adaptation), and seamless integration with private infrastructure.
• Fireworks AI : An inference and fine-tuning platform geared toward production environments. It allows enterprises to fine-tune smaller open-source models with proprietary data inside secure environments and serve them with ultra-low latency.
• SiliconFlow : An emerging infrastructure platform optimized for high-performance open-source model hosting, custom fine-tuning, and efficient local-or-cloud execution workflows.
• Hugging Face AutoTrain : While part of a larger ecosystem, Hugging Face provides zero-code and automated training interfaces that allow enterprises to securely fine-tune smaller models on private datasets without heavy MLOps overhead.

If you'd like, I can:

• Explain how QLoRA and quantization work for on-device deployment
• Compare cloud fine-tuning vs. local GPU hardware requirements

The ecosystem for customizing and fine-tuning small models (Small Language Models or SLMs) for private, on-premise, or on-device deployment relies heavily on specialized ML infrastructure platforms, parameter-efficient fine-tuning (PEFT) frameworks, and optimization toolkits.

While major tech giants provide native hardware runtimes (like Apple's Core ML / MLX and Google's MediaPipe / AI Edge ), several dedicated platforms and venture-backed startups specialize in streamlining the fine-tuning, adaptation, and private serving lifecycle for lean models:

• Predibase - A managed platform built by the creators of Ludwig and LoRAX. It specializes in ultra-efficient Parameter-Efficient Fine-Tuning (PEFT) using LoRA and QLoRA, allowing teams to fine-tune open-source small models (like Llama, Mistral, or Phi families) and efficiently serve dozens of custom adapters simultaneously on shared infrastructure.
• Hugging Face (Enterprise/Tuning Ecosystem) - While not strictly a traditional "startup" anymore, Hugging Face provides the foundational enterprise tooling (PEFT, TRL , and bitsandbytes ) that almost all on-device and private fine-tuning pipelines depend on. They actively bridge the gap for exporting fine-tuned checkpoints into mobile-ready quantization formats.[1]
• Baseten - Frequently utilized by ML teams for high-performance private and on-premise model deployment. Baseten gives teams granular control over model execution, enabling optimized custom-tuned small models to run with minimal latency inside secure private clouds or enterprise VPCs.
• Anyscale - Built on Ray, Anyscale powers scalable private fine-tuning and distributed training workloads. It allows enterprise engineering teams to run localized or private data-center fine-tuning jobs on open-source SLMs without sending data to third-party public LLM APIs.
• Modal - A serverless cloud execution platform that has become heavily adopted by engineering teams to run ad-hoc, heavy fine-tuning scripts and quantization jobs for smaller open-source models before packing them down for local or edge deployment.

Key Technical Approaches These Startups Use

1. LoRA & QLoRA: Instead of retraining full model weights, platforms rely on low-rank adaptation to make fine-tuning small models (1B–8B parameters) cheap enough to run on single consumer-grade GPUs or local hardware.[1]
2. Quantization-Aware Training (QAT) & Post-Training Quantization: Compressing models down to 4-bit or 8-bit precision so they fit within the unified memory or NPU constraints of edge hardware like iPhones, Androids, or local IoT gateways.

If you'd like to narrow this down, tell me:

• Are you targeting mobile/edge hardware (iOS/Android) or a private on-premise server?
• What size range of models (e.g., sub-1B vs 3B–8B parameters) are you looking to adapt?

I can give you a more targeted stack recommendation or architecture workflow.

A number of startups are building tooling around small language models (SLMs), fine-tuning, quantization, and private/on-device deployment. The space is still emerging, so some are full platforms while others are more like specialized infrastructure providers.

Fine-tuning platforms that export small models

• Ertas AI — A no-code/low-code platform for fine-tuning open models and exporting them for local deployment. It focuses on taking a dataset → tuned model → GGUF/local runtime workflow, with support for running models via tools such as llama.cpp and local model runtimes.
• Forgelet — Targets app developers who want task-specific small models without building ML pipelines. It describes a workflow where users define a task, generate or provide data, fine-tune a compact model, and export it for phones, servers, or embedded devices.
• Gerbil SDK — Provides tooling to train small specialized models from user data and keep the resulting weights. Its positioning is around replacing repeated frontier-model API calls with smaller task-specific models that can run locally or anywhere.
• SmaLLM — Focuses on domain-trained small language models, including fine-tuning/distillation pipelines and deployment across private cloud, edge, mobile, and browser environments.

On-device/private AI specialists

• Smelt.works — Focuses specifically on fine-tuning compact models for mobile and desktop use cases. It highlights 1–3B parameter models for mobile, 7–13B for desktop, and fully local inference.
• Flower Intelligence — Works on local-first AI with on-device inference and privacy-preserving approaches. Its roadmap includes local and federated fine-tuning so models can improve using device-local data.
• Icosa Computing — Builds private local AI workflows around custom models that users can train/tune and run on their own hardware. Its Zeno product is positioned around local model usage and private AI.

Companies doing private deployment / enterprise customization

• Vravio Inc — Provides engineering services around fine-tuning open-weight models (such as Gemma, Llama, and similar models) and deploying them privately/on-premises.
• Callstack — More of an AI engineering services company than a pure startup platform, but it works on adapting models with LoRA and deploying optimized AI directly on iOS/Android devices.

Adjacent infrastructure worth watching

• DeepEdge AI — Works on edge AI systems, including model compression, on-device learning, and deployment across constrained hardware.

How I’d categorize the market

A key trend is that teams increasingly fine-tune rather than train from scratch: use LoRA/QLoRA adapters, distill from larger models, quantize (GGUF, 4-bit, etc.), then ship a smaller model that can run inside a phone app, browser, desktop app, or private server.

If you’re looking specifically for venture-backed startups with funding/traction (rather than all emerging vendors), the list becomes much shorter.

Several specialized startups and platforms help engineering teams fine-tune small models (SLMs) and compress them for on-device, edge, or private enterprise deployment.

If you have a specific target platform in mind—such as iOS/Android, embedded hardware , or a local air-gapped server —let me know and I can narrow down the best toolchain for your hardware constraints.

Yes. If you mean startups whose core proposition is helping a team turn a general/open model into a smaller, specialized model that can run on-device, at the edge, or inside a private environment, there’s a fairly interesting emerging group.

Most directly aligned

• forgelet.tech — Probably the closest match. It handles dataset generation, fine-tuning, and export, targeting roughly 270M–3B parameter models for phones and other constrained hardware. It exports to GGUF, LiteRT, and Core ML, with fully offline inference.
• smelt.works — Focuses specifically on fine-tuning compact models for users' devices: roughly 1–3B for mobile and 7–13B for desktop, with LoRA/full-weight training and zero cloud inference.
• small-lm.com — More enterprise-oriented: captures production traces, uses teacher models for synthetic data/critique, fine-tunes and distills specialist SLMs, then deploys them in private cloud, edge, mobile, or browsers.
• sabrresearch.com — Builds domain-specific small language models and deploys them on-prem or air-gapped, including transferring the resulting model under a commercial license. This is more services/enterprise delivery than developer tooling.
• vravio.com — Fine-tunes open models such as Gemma, Llama and gpt-oss with LoRA/QLoRA, then delivers them for VPC/on-prem deployment. Again, more AI engineering/services than a pure platform.

Adjacent companies worth watching

• Flower — Particularly interesting if you're thinking about federated/local fine-tuning rather than simply training centrally and exporting a model. Its Flower Intelligence work targets local inference, with local and federated fine-tuning on the roadmap.
• Callstack — Helps teams adapt models with LoRA and optimize them for iOS/Android using runtimes such as ExecuTorch, MediaPipe, llama.cpp and MLX. More of an engineering/implementation partner than a fine-tuning SaaS.
• Icosa Computing — Its LMShop is aimed at letting users bring documents, build/tune a custom model, compare it against frontier models, and deploy it locally. Its Zeno product is explicitly local-first.
• AGI, Inc. — Building its own small action model (AGI-0), with deployment across cloud, on-prem and edge and an explicit goal of getting small enough to run directly on devices. It's more of a model/platform company than a generic fine-tuning provider.

How I'd segment the market

The particularly interesting white space is between Forgelet/Smelt and SmaLLM/Sabr: a product where an enterprise team can take its proprietary data, automatically create a high-quality fine-tuning set, distill a 0.5–4B model, benchmark it against its frontier-model baseline, quantize/compile it for a specific device, and then ship the resulting weights into an air-gapped/on-device runtime.

That looks like a fairly distinct category from traditional LLM fine-tuning platforms such as generic cloud training APIs.